Re: Tepid: selective index updates for heap relations

From: "Greg Burd" <greg(at)burd(dot)me>
To: "Alexander Korotkov" <aekorotkov(at)gmail(dot)com>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Tepid: selective index updates for heap relations
Date: 2026-07-11 22:04:34
Message-ID: 3dc5d57e-6c77-4713-b97e-bf0a5a891226@app.fastmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I should have checked and rebased before I posted v59, apologies. Here's v60 which should be clean against 1c4b1de8885.

Also fixed the following oversights:

1. CLUSTER / VACUUM FULL data loss + index staleness on SIU tables — the heap rewrite left the stale HEAP_INDEXED_UPDATED bit on flattened tuples, so readers ran the staleness test against a now-meaningless inline bitmap and dropped live rows (and returned nothing via index). Fixed by clearing the marker in reform_tuple. Hit any SIU table after a routine VACUUM FULL.

2. amcheck heapallindexed false positive — its bloom-filter fingerprint of a flagged leaf TID never matched the plain heap TID it re-derives, raising a spurious "lacks matching index tuple." Fixed by stripping the marker while fingerprinting in verify_nbtree.

3. Tooling
- pageinspect 1.14 — bt_page_items now reports the real offset in ctid/htid (was showing an inflated offset) and adds a hot_indexed column exposing the marker.
- Sentinel-safe strip — ItemPointerGetOffsetNumber/ItemPointerCompare strip bit 14 but leave the SpecToken/MovedPartitions offset sentinels untouched (a self-inflicted regression caught during development).

best.

-greg

Attachment Content-Type Size
v60-0001-Add-tests-to-cover-a-variety-of-heap-HOT-update-.patch text/x-patch 45.4 KB
v60-0002-Identify-modified-indexed-attributes-in-the-exec.patch text/x-patch 61.4 KB
v60-0003-Add-the-HOT-indexed-on-disk-format-inline-attr-b.patch text/x-patch 12.8 KB
v60-0004-Add-HOT-indexed-updates-selective-index-maintena.patch text/x-patch 230.8 KB
v60-0005-Fix-a-BitmapAnd-BitmapOr-false-negative-on-HOT-i.patch text/x-patch 40.1 KB
v60-0006-Collapse-dead-HOT-indexed-chains-to-xid-free-stu.patch text/x-patch 51.8 KB
v60-0007-Teach-amcheck-to-recognize-HOT-indexed-chains-an.patch text/x-patch 17.2 KB
v60-0008-Add-HOT-indexed-statistics-and-the-comprehensive.patch text/x-patch 195.8 KB
v60-0009-Gate-HOT-indexed-updates-on-the-logical-replicat.patch text/x-patch 114.9 KB
v60-0010-DO-NOT-MERGE-Add-a-HOT-SIU-benchmark-harness.patch text/x-patch 39.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Scott Ray 2026-07-12 00:22:00 pg_xmin_horizon: a system view of everything pinning the xmin horizon
Previous Message Joel Jacobson 2026-07-11 21:52:13 Re: Key joins