| From: | Greg Burd <greg(at)burd(dot)me> |
|---|---|
| To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | Peter Geoghegan <pg(at)bowt(dot)ie>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Nathan Bossart <nathandbossart(at)gmail(dot)com> |
| Subject: | Re: Tepid: selective index updates for heap relations |
| Date: | 2026-11-17 20:02:23 |
| Message-ID: | avyw8yeme1-2kBl2@floki |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
I appreciate the careful read; here is v83, rebased on 70c4a57b64d.
An index build put its entry for a live HOT-indexed row at that row's
own TID, for every index, while pruning assumes the only entries at a
heap-only member are the ones its own UPDATE planted. So a member could
be reclaimed with a live entry pointing at it. CREATE INDEX on b
reproduces it as you said, and so do REINDEX INDEX, REINDEX TABLE and
both CONCURRENTLY forms on any index the chain did not change. REINDEX
of the changed index lost rows another way: when the live member's own
update changed no indexed column, that member was reclaimed outright and
took the rebuilt entry with it. In every case an index scan stopped
returning the row and amcheck's heapallindexed check failed. Every
posted version had this.
0016 now anchors each live row, per index, at the latest chain member
that changed one of that index's attributes, or at the root if none
did.
That is where an UPDATE would have put the entry, so pruning's rule
holds for built entries too and for readers. CIC's validate pass uses
the same anchor. 0018 adds the three cases to pageinspect's
hot_indexed_updates test; they fail without the fix.
I also wrote a small fuzzer that mixes updates of different columns with
VACUUM, CREATE/DROP/REINDEX [CONCURRENTLY], and compares every index
against a seqscan after each step and that's helped find a few more
issues.
On the stale-entry flag, I agree that auditing callers doesn't scale so
I'm flipping the default. The heap AM now skips flagged entries
inside its index scan, and a caller sees one only if it asks with a
new scan option, SO_INDEX_KEEP_STALE.
Only check_exclusion_or_unique_constraint() asks: with a dirty snapshot,
an in-progress update's row can be reachable through its old entry
before its new entry is inserted, so it has to see that entry and
compare the row's current key. The checks in IndexScan, CLUSTER,
REPACK, the FK fast path and the replication lookups are gone, and new
callers get the safe behavior without knowing about any of this.
On index_delete_check_htid() upon further review I agree that relaxing
all three checks for every table was too broad. So v83 restores
master's three checks for every relation, with one exception: an entry
may point at a heap-only tuple that carries HEAP_INDEXED_UPDATED. That
is narrower than tracking it per table, and a table that never takes one
of these updates keeps every check.
On the visibility map, I agree but I'm stuck, and I have no fix in v83.
I measured it on 100,000 rows across 1,924 pages, with a HOT-indexed
update of one row in fifty. After three vacuums, no page is
all-visible, and an index-only scan over all rows does 102,000 heap
fetches. Deleting those rows and vacuuming brings all 1,924 pages back.
The stale entry keeps the root's TID live, so VACUUM can't remove it,
and the page stays out of the map. I think the fix is the one you
suggest, VACUUM removing a stale entry from just the index it is stale
in, but that is a significant new piece of work. Until something like
that exists this is a trade I don't expect anyone to accept as a
default.
I'll consider posting 0001-0008 on their own in a new thread, as you
suggest. I'd been thinking about that as well.
best.
-greg
| Attachment | Content-Type | Size |
|---|---|---|
| tepid-v83.tgz | application/gzip | 273.1 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Greg Burd | 2026-11-17 20:11:48 | Re: Keep TIDBitmap pagetable memory across bitmap scans |
| Previous Message | Greg Burd | 2026-11-17 19:53:31 | Re: Let an ordering index scan hand its ORDER BY value to the target list |