Re: Compression of bigger WAL records

From: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
To: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
Cc: wenhui qiu <qiuwenhuifx(at)gmail(dot)com>, Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Compression of bigger WAL records
Date: 2026-03-09 17:07:59
Message-ID: 4D8E3280-632B-4C6C-A728-835B7FDE6325@yandex-team.ru
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On 16 Jan 2026, at 21:17, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>
> That's a very good idea! We don't need to replace current behavior, we can just complement it.
> I'll implement this idea!

Here's the implementation. Previously existing buffers are now combined
into single allocation, which is GUC-controlled (you can add more memory).

However, now this buffer is just enough to accommodate most of records...
So, maybe we do not need a GUC at all, because keeping it minimal (same
consumption as before the patch) is just enough.

Now the patch essentially have no extra memory footprint, but allows to
save 25% of WAL on index creation (in case of random data).

User can force FPI-only compression by increasing wal_compression_threshold
to 1GB.

The decision chain is now a bit complicated:
- assemble record without compression FPIs
- try whole record compression
- if compression enlarged record fallback to FPI compression
I think the case can be simplified to "Try only one compression approach that
is expected to work, if not - insert uncompressed".

What do you think?

Best regards, Andrey Borodin.

Attachment Content-Type Size
v6-0001-Add-whole-record-WAL-compression-alongside-FPI-co.patch application/octet-stream 54.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Hüseyin Demir 2026-03-09 18:18:05 Re: client_connection_check_interval default value
Previous Message Bryan Green 2026-03-09 17:06:28 Re: Avoid multiple calls to memcpy (src/backend/access/index/genam.c)