Re: Compression of bigger WAL records

Hi, Andrey

On Mon, 09 Mar 2026 at 22:07, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>> On 16 Jan 2026, at 21:17, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>>
>> That's a very good idea! We don't need to replace current behavior, we can just complement it.
>> I'll implement this idea!
>
> Here's the implementation. Previously existing buffers are now combined
> into single allocation, which is GUC-controlled (you can add more memory).
>
> However, now this buffer is just enough to accommodate most of records...
> So, maybe we do not need a GUC at all, because keeping it minimal (same
> consumption as before the patch) is just enough.
>
> Now the patch essentially have no extra memory footprint, but allows to
> save 25% of WAL on index creation (in case of random data).
>
> User can force FPI-only compression by increasing wal_compression_threshold
> to 1GB.
>
> The decision chain is now a bit complicated:
> - assemble record without compression FPIs
> - try whole record compression
> - if compression enlarged record fallback to FPI compression
> I think the case can be simplified to "Try only one compression approach that
> is expected to work, if not - insert uncompressed".
>
> What do you think?
>
>

Thanks for updating the patch. It seems a rebase is needed.

$ git am ~/v6-0001-Add-whole-record-WAL-compression-alongside-FPI-co.patch
Applying: Add whole-record WAL compression alongside FPI compression
error: patch failed: src/backend/access/transam/xloginsert.c:115
error: src/backend/access/transam/xloginsert.c: patch does not apply
Patch failed at 0001 Add whole-record WAL compression alongside FPI compression
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

> Best regards, Andrey Borodin.
>
> [2. text/x-diff; v6-0001-Add-whole-record-WAL-compression-alongside-FPI-co.patch]...

--
Regards,
Japin Li
ChengDu WenWu Information Technology Co., Ltd.