Re: Compression of bigger WAL records

Hi Michael, Zsolt!

On 14 Oct 2026, Michael Paquier wrote:
> In short, applied the WAL insert part.

Thank you! Attached is v12: whole-record compression (0001), a separate
pg_walinspect update (0002), and WIP streams (0003).

Context reuse removes per-image setup cost. These patches target
redundancy between images in a record and between successive records.

Zsolt, you are right about incompressible records. 0001 now falls back
unless compression saves space including its header. Streams in 0003
still allow individual records to grow to preserve history for later
records.

Following Nikolay Samokhvalov's off-list review [0], FPI-only settings
no longer allocate the large whole-record workspaces. Those are prepared
outside critical sections only for settings that can use them.
A writer also keeps one stream compressor across lease changes,
resetting it when acquiring a different slot. My earlier
one-compressor-per-writer description was not true of v11.

On the statistics question raised by both Zsolt and Nikolay, pg_waldump
and pg_walinspect now expose physical sizes separately from decoded
sizes. I kept wal_fpi_bytes as image bytes before whole-record
compression and documented that it can exceed physical wal_bytes.
There is no exact way to split a jointly compressed payload into image
and non-image bytes.

0003 is still WIP: reader memory and the cost of rewinding to a stream
reset boundary remain open trade-offs.

Best regards, Andrey Borodin.

[0] https://github.com/NikolayS/postgres/issues/99#issuecomment-6068279914