Re: Stop masking visibility information during WAL consistency checking

From: Zhijie Hou <houzhijie22(at)gmail(dot)com>
To: Melanie Plageman <melanieplageman(at)gmail(dot)com>, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org, surya poondla <suryapoondla4(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Michael Paquier <michael(at)paquier(dot)xyz>
Subject: Re: Stop masking visibility information during WAL consistency checking
Date: 2026-11-21 12:18:45
Message-ID: 18f55c4f-be92-4bc2-bb1d-9d21641a75ae@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On Sat, Nov21, 2026 at 07:30 AM Melanie Plageman wrote:
> On Fri, Nov 20, 2026 at 6:17 PM Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com> wrote:
> >
> > > Stop masking PD_ALL_VISIBLE, as doing so can only conceal real bugs.
> >
> > Claude pointed out that wal_consistency_checking can trigger an
> > error with 0002. This uses very specific flags, but both
> > full_page_writes=off and disabled data checksums are supported, so
> > it seems like a plausible scenario.
>
> Ah, right, I forgot that when we only sets the VM bits and both
> checksums/wal_log_hints are off, we can set PD_ALL_VISIBLE without
> advancing the heap page LSN. So, I suppose we can't do 0002. I'll
> replace it with a replacement of the comment explaining why we have to
> mask PD_ALL_VISIBLE. Thanks!

I analyzed this bit more, and think the same issue can also cause a
false positive in 0001. visibilitymap_prepare_truncate() also modifies
the VM page without advancing pd_lsn when checksums and wal_log_hints
are off (as in Zsolt's second test case). If the server crashes
immediately after that VM page is flushed to disk, recovery sees the
post-truncation VM page contents with the pre-truncation pd_lsn. When
replaying the last VM-modifying record before the truncation,
wal_consistency_checking fails because the on-disk page does not match
the image logged before the truncation. We could fix this by
unconditionally WAL-logging the VM page there, which advances pd_lsn as
well. (But the fix looks like separate change worth some discussions and
some consensus)

And I also noticed two other cases (when full_page_writes is off) that
cause similar failures:

1. If the server crashes after dropping a table, the VM file has already
been unlinked. During recovery, replaying an earlier XLOG_HEAP2_PRUNE_*
record passes RBM_ZERO_ON_ERROR when reading the VM page, which creates
a new zero-filled page if the file does not exist. After replaying that
record's VM change onto the new page, its contents still do not match
the logged image because any other bits previously set on that VM page
were lost.

2. If the VM fork is truncated and then re-extended, vm_extend() writes
an all-zero page on disk while the page header is initialized only in
shared_buffers, leaving an all-zero page on disk. If the server crashes
immediately after extending the fork, recovery runs the consistency
check against that all-zero page and fails (note that this issue also
exists on master for zero-extended heap pages, where mask_unused_space()
fails on pd_lower == 0). To fix this, we could skip the consistency
check when the page is new (PageIsNew()) and the current WAL record does
not initialize it.

--
Best Regards,
Zhijie Hou

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2026-11-21 12:37:32 Re: Changing the state of data checksums in a running cluster
Previous Message Henson Choi 2026-11-21 12:11:44 Re: [PATCH v1] Preserve implicit composite casts in ROW deparsing