Re: Proposal: Adding compression of temporary files

From: Filip Janus <fjanus(at)redhat(dot)com>
To: lakshmi <lakshmigcdac(at)gmail(dot)com>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, zsolt(dot)parragi(at)percona(dot)com
Subject: Re: Proposal: Adding compression of temporary files
Date: 2026-01-25 11:57:05
Message-ID: CAFjYY+J+TN-Ovi=ToGG4CcJweY_a6Hb6xL67fMESqKpT=FXuJQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Fixed spacing in the patch.

-Filip-

pá 23. 1. 2026 v 17:40 odesílatel Filip Janus <fjanus(at)redhat(dot)com> napsal:

> Hi all,
> Thanks for the feedback and the provided patch.
> I've addressed your findings and proposals. Lakshmi's documentation patch
> was incorporated.
>
> -Filip-
>
>
> st 21. 1. 2026 v 7:30 odesílatel lakshmi <lakshmigcdac(at)gmail(dot)com> napsal:
>
>> HI all,
>> While testing the temp file compression patch,noticed that the new
>> temp_file_compression GUC isn't documented yet.I put together a small docs
>> patch to add a short description and clarify that the effect of compression
>> depends on the workload(for example ,hash join spills may not show visible
>> size reduction due to fixed_size chunks).
>>
>> patch is attached.Happy to adjust the wording if needed.
>> thanks,
>> lakshmi
>>
>> On Tue, Jan 20, 2026 at 4:21 PM lakshmi <lakshmigcdac(at)gmail(dot)com> wrote:
>>
>>> Hi Filip,
>>>
>>> I tested both patches on current master using git am -3 .They apply
>>> cleanly,build fine,and the temp_file _compression GUC works as expected.
>>> Query results are unchanged.
>>>
>>> For hash join spill test,temp files were created as expected,but the
>>> logged size were same for no,lz4,and pglz,which seems consistent with
>>> fixed-size fileset chunking.It might be helpful to briefly note this in the
>>> documentation to avoid confusion.
>>>
>>> Thanks for working on this .
>>> best regards,
>>> lakshmi
>>>
>>> On Tue, Jan 20, 2026 at 4:10 AM Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>
>>> wrote:
>>>
>>>> Hello!
>>>>
>>>> I tried to review the code. It compiled, the test suite passed.
>>>>
>>>> I noticed two typos:
>>>>
>>>> buffile.c:77 - "Disaled"
>>>> buffile.c:133 - "mathods"
>>>>
>>>> And a few other small findings:
>>>>
>>>> buffile.h:35 and buffile.c:63 - same constants defined first as an
>>>> Enum and then as #defines - code builds properly without the defines.
>>>>
>>>> buffile.c:121 - compress_tempfile is defined, set to false at :167,
>>>> but never used otherwise
>>>>
>>>> guc_tables.c:470 - the comment says that pglz isn't supported yet, but
>>>> we have a value for it, and I see support for it in the code
>>>>
>>>> buffile.c:659: (and at other places) if USE_LZ4 is undefined, the
>>>> codepath doesn't do anything. I think these ifdefs should follow how
>>>> other compression code works, such as wal compression where there's an
>>>> #else path with elog(ERROR, ...)
>>>> Similarly, maybe there should be an explicit TEMP_NONE_COMPRESSION
>>>> branch that does nothing, and the default branch should be an error?
>>>>
>>>> buffile.c:265: If seek isn't supported/limited, shouldn't there be at
>>>> least an assertion about it in BufFileSeek? And tell isn't mentioned,
>>>> but it seems to me that tell also doesn't work properly.
>>>>
>>>

Attachment Content-Type Size
0001-Add-transparent-compression-for-temporary-files.patch application/octet-stream 20.8 KB
0002-Add-regression-tests-for-temporary-file-compression.patch application/octet-stream 127.6 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Alexander Lakhin 2026-01-25 12:00:00 Re: [BUG] [PATCH] pg_basebackup produces wrong incremental files after relation truncation in segmented tables
Previous Message Peter Eisentraut 2026-01-25 10:34:38 Re: alignas (C11)