Re: Reduce WAL volume for heap tuple hint bits

Hi Andrey,

On Wed, Aug 5, 2026 at 5:18 PM Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>
> Hi hackers,
>
> Heap tuple visibility hint bits avoid repeated transaction status
> lookups. With wal_log_hints enabled, PostgreSQL currently logs a
> full-page image for the first hint bit change to a page after each
> checkpoint. This can generate considerably more WAL than the hint bits
> themselves require.
>
> My main motivation comes from our PostgreSQL fork with compute/storage
> separation, where this WAL volume is costly. The same overhead also
> occurs in vanilla clusters with wal_log_hints enabled.
> The concept is mostly viable when torn pages are not a problem.
>
> The attached patch records heap tuple offsets and visibility hint bits
> in a compact WAL record when checksums are disabled. WAL replay applies
> the hints on standbys. The record does not advance the page LSN, so a
> later ordinary change still generates the FPI required by
> full_page_writes. Checksummed pages continue to use FPI_FOR_HINT.
>
> For the first scan of one million rows, WAL fell from 36.8 MB to 4.2 MB
> for an int-only table and from 134.7 MB to 4.9 MB for a
> pgbench_accounts-shaped table. With wal_compression=lz4, WAL fell from
> 13.9 MB to 4.2 MB and from 16.3 MB to 4.9 MB, respectively. About 3.3x in
> both cases.
>
> PFA.
>

Thanks for working on this — the WAL overhead of wal_log_hints is a
real pain point, and the
compact record for the checksums-off case is a nice win.

I read v1 and tested it, and the correctness-critical parts hold up:
not advancing the page LSN correctly
preserves the FPI for the next ordinary change and keeps replay
idempotent, and the DataChecksumsNeedWrite()
gating leaves no torn-page window. A crash-recovery run with
wal_consistency_checking = 'all' showed no inconsistencies,
the 056 test passes, and pg_waldump confirms the ~655-byte records vs
an 8 KB FPI.

One design question before going further: why a new resource manager
(RM_HEAP_HINT) rather than a new info opcode
under the existing heap rmgr? A dedicated rmgr spends one of the
limited RM_*_ID slots, and folding it into heap would keep
heap WAL together. Is there a reason I'm missing?

Thanks again!

>
> Best regards, Andrey Borodin.
>

--
Regards,
Ewan Young