| From: | Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
|---|---|
| To: | vignesh C <vignesh21(at)gmail(dot)com> |
| Cc: | shihao zhong <zhong950419(at)gmail(dot)com>, Greg Burd <greg(at)burd(dot)me>, pgsql-hackers(at)postgresql(dot)org, pgsql-hackers(at)lists(dot)postgresql(dot)org, michael(at)paquier(dot)xyz, pgsql(at)j-davis(dot)com, exclusion(at)gmail(dot)com |
| Subject: | Re: Hash index: stop registering pin-only bucket pages in WAL (remove REGBUF_NO_CHANGE) |
| Date: | 2026-11-11 11:49:45 |
| Message-ID: | CAA4eK1LiqA_pgnyTEZtGDXTCDBS801x3X-4r8+iJzN0-ZKp7XA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Wed, Nov 11, 2026 at 2:59 PM vignesh C <vignesh21(at)gmail(dot)com> wrote:
>
> On Wed, 11 Nov 2026 at 08:33, shihao zhong <zhong950419(at)gmail(dot)com> wrote:
> >
>
> Thanks for the report. I was able to reproduce this on HEAD.
> The attached test
> (src/test/recovery/t/068a_hash_torn_primary_page_delete_first.pl) does
> the following, with data checksums enabled: a) Builds a hash index
> with all keys in one bucket, consisting of a primary page P and an
> overflow page, and takes a checkpoint. b) Runs VACUUM to remove dead
> entries from the overflow page only. This writes an XLOG_HASH_DELETE
> record for the overflow page without modifying P. c) Runs a second
> VACUUM to remove dead entries from P. This is the first modification
> to P since the checkpoint, so the corresponding WAL record includes a
> full-page image (FPI) of P. d) Crashes the server, overwrites the
> second half of P in the data file with random bytes to simulate a torn
> write, and restarts the server.
>
> Crash recovery then fails while replaying the first record (the DELETE
> for the overflow page), before reaching the record containing the FPI
> of P:
> FATAL: invalid page in block 7 of relation "base/5/16387"
> CONTEXT: WAL redo at 0/017745E0 for Hash/DELETE: ... is_primary F;
> blkref #1: ... blk 10 FPW
>
> During redo, we read P only to acquire the cleanup lock, using
> RBM_NORMAL. The read therefore fails checksum verification. The test
> fails in the same way without the patch, where P is accessed through
> block 0, and with the patch, where it is accessed through
> bucket_blkno.
> Since we only need P to acquire the cleanup lock and don't actually
> need its contents, I think we should read it using RBM_ZERO_ON_ERROR
> in hash_xlog_lock_bucket(). This would zero the buffer instead of
> raising an error when a torn page is encountered, allowing the FPI
> later in the WAL to restore the correct contents.
>
> Do you think RBM_ZERO_ON_ERROR is the right approach here, or is there
> a better solution to fix this?
>
The code comments related to RBM_ZERO_ON_ERROR states following:
"RBM_ZERO_ON_ERROR is like the normal mode, but if the page header is
not
valid, the page is zeroed instead of throwing an error. This is
intended for non-critical data, where the caller is prepared to repair
errors.". And, I see RBM_ZERO_ON_ERROR mode is only used for VM and
FSM pages which don't contain user-data. So, it doesn't sound like a
good idea to use RBM_ZERO_ON_ERROR.
Can we explore guarding the read/lock of bucket_page during replay by
HotStandbyActiveInReplay()? IIRC, the cleanup lock is required during
replay to prevent concurrent scans from reading incorrect data. So,
once the consistency is reached, we should be okay to read/lock the
primary page.
I feel we should also explore the idea of locking the bucket page even
when we are registering without it being changed to avoid this
problem. I'll also think about it.
--
With Regards,
Amit Kapila.
| From | Date | Subject | |
|---|---|---|---|
| Previous Message | ZizhuanLiu X-MAN | 2026-11-11 11:37:02 | Re: Incorrect selectivity estimates for NOT on nullable boolean columns |