Re: amcheck: add index-all-keys-match verification for B-Tree

From: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
To: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
Cc: Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Wenbo Lin <linwenbo1994(at)gmail(dot)com>
Subject: Re: amcheck: add index-all-keys-match verification for B-Tree
Date: 2026-11-16 11:25:29
Message-ID: CAKZiRmzoGu3E72omHvw=yLs24oXf3rpVCPYrDNJjQjcSx1XB3Q@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Nov 12, 2026 at 2:09 PM Jakub Wartak
<jakub(dot)wartak(at)enterprisedb(dot)com> wrote:
>
> On Sun, Nov 8, 2026 at 8:23 AM Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
> >
> > Hi Jakub,
>
> Hi Andrey,
>
> > On 5 Oct 2026, Jakub Wartak wrote:
> > > still follows LP_REDIRECT/HOT
> >
> > PFA v7 with your API fixup folded into 0001 and the old 0002/0003 merged.
> [..]
>
> Cool!
>
> I've found two missing things and I'm attaching as one-off patches for
> consideration and some other third small thing:
> 1) pg_amcheck was missing --indexallkeysmatch support
> 2) we could check if btree isn't pointing directly to HOT (just in case)
>
> Also I've been throwing this code under a bus a lot at 100MB and 10GB+ scales
> with over 10k TPS and vacuum hammering it like crazy for dozen of minutes
> (with threshold/scales='1' and with high concurrency pgbench), and I couldn't
> find a single false positives being generated: both on primary and standby
> with parallel pg_amchecks -j X --indexallkeysmatch , on standby of with with
> hot_standby_feedback to avoid rep conflicts).
>
> Also I've made braindump about and come to the conclusion that perhaps the
> below table could be part of docs or just having it in the code would be
> useful, thoughts on this? (perhaps please double check me on this if we
> want to have it in docs)
>
> corruption-kind b_i_c() b_i_parent_c()
> lost heap segment/TID beyond end of heap yes* yes*
> index<->heapam key mismatch yes yes
> TID points to LP_UNUSED no yes
> TID offset beyond the page line pointer array no yes
> TID points directly at HOT (w/o LP_REDIRECT) no yes
>
> * = even without indexallkeysmatch
> b_i_c() b_i_parent_c() stand for shortcuts of SQL function names.
>
> Anyway, I find the tradeoff betwen lock mode (SL vs ASL) and being able to
> idenitify corrupted things quite good as it is now. Perhaps, we should set
> it as ready for commit now?
>
> Perhaps, qq, should we care if there are other tableAM used with btree
> somehow? (I haven't tried that as I'm struggling to find one that works with
> master/PG20)

This is just to answer myself on the last question (tableAM, got one non-
standard one running on PG20): bt_report_dangling_index_entry() quits if relam
!= HEAP_TABLE_AM_OID, however during the further tests on v7 against
non-heapAM, I've got false errors from bt_check_every_level(). I think we can
assume that TID is a physical location for heapAM, but for other tableAMs this
might be NOT true (they can give TID whatever they like from I can see).

So fix the beyond-EOF check so that it check this only for heapAM. Attached is
set of patches as it was (v7), plus the new one: 0007. @Andrey: if You are happy
with them, they should be just squashed (I didn't want to do it, just to
show the diffs over time). Anyway, the rest of the question about doc probably
and should we switch the CF entry to 'ready for commiter' persists.

-J.

Attachment Content-Type Size
v16112026-0002-amcheck-detect-dangling-B-Tree-references-.patch text/x-patch 17.7 KB
v16112026-0003-doc-describe-indexallkeysmatch-verificatio.patch text/x-patch 7.6 KB
v16112026-0001-amcheck-add-indexallkeysmatch-verification.patch text/x-patch 35.0 KB
v16112026-0004-pg_amcheck-add-indexallkeysmatch-option.patch text/x-patch 7.8 KB
v16112026-0005-amcheck-verify-against-dangling-btree-poin.patch text/x-patch 1.6 KB
v16112026-0006-mention-github-action-runners-CI.patch text/x-patch 1.1 KB
v16112026-0007-verify_heapam-make-beyond-EOF-TID-check-he.patch text/x-patch 1.6 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2026-11-16 11:25:59 Re: [PATCH] doc: Clarify pg_checksums --filenode mode restriction
Previous Message Ajin Cherian 2026-11-16 11:24:43 Re: [PATCH] Preserve replication origin OIDs in pg_upgrade