Re: Tepid: selective index updates for heap relations

From: Greg Burd <greg(at)burd(dot)me>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: Peter Geoghegan <pg(at)bowt(dot)ie>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Nathan Bossart <nathandbossart(at)gmail(dot)com>
Subject: Re: Tepid: selective index updates for heap relations
Date: 2026-11-06 18:42:44
Message-ID: au4fcF2KSwHRnhrl@floki
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Here is v80, rebased on f8cabd1f175. It fixes two bugs I found in v79,
both in 0013.

A HOT-indexed update leaves the old index entry in place, and that entry
still leads to the updated row. The table AM flags such an entry
(xs_entry_needs_recheck) and every consumer has to drop it. Two
consumers did not:

- The foreign key fast path (ri_FastPathProbeOne) took the first row
the index returned. After UPDATE parent SET k = 5000 WHERE k = 5, an
INSERT into the child with k = 5 succeeded and left an orphan row.
That happened for a referenced primary key and for a unique key. The
SPI path, which ALTER TABLE ... ADD FOREIGN KEY uses, was correct.

- REPACK (CONCURRENTLY) finds the row to change through the same kind
of entry when it replays concurrent changes. If a key moved and was
then reused, the replay matched the wrong row and REPACK failed with
a duplicate key error. Master runs the same sequence without error.

Both now skip flagged entries and keep scanning, as IndexScan and the
logical replication lookups already did. 0018 adds an FK test (old
primary and unique key values are rejected, new values are found, and a
key changed away and back is still found), and 0016 adds an isolation
test for the REPACK case.

I went through the other index scan callers. Catalog scans
(systable_getnext) never see these entries, since catalogs never take
HOT-indexed updates, and get_actual_variable_endpoint() only feeds
estimates. If I've missed a caller, that is the review I most need
here. Every caller has to opt in to the check, which is Chao's point
from July, and so far my only answer is auditing them.

Only 0013, 0016 and 0018 changed since v79; the rest differ only by the
rebase. 0013 and 0016 also lost a duplicated #include and two stale
comments, and 0016's comments on index_delete_check_htid() now say what
the code does: the relaxed corruption checks apply to every non-catalog
relation, since nothing records whether a table has ever taken a
HOT-indexed update. Narrowing that would need such a record, and I'd
like opinions on whether losing those checks is acceptable.

cfbot had one failure on v79, on Oct 31: an ecpg TIMEOUT on Windows (VS
slice 1; the other 200 tests passed). Four other CF entries hit the
same ecpg timeout on Windows that week, and the Nov 3 retest of v79
passed everywhere, so I don't think it is this series.

0001-0008 still stand on their own,

-greg

Attachment Content-Type Size
tepid-v80.tgz application/gzip 271.1 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Burd 2026-11-06 18:48:04 Re: UNDO with FILEOPS, and the FLUX table AM
Previous Message Greg Burd 2026-11-06 18:36:04 Re: Let an ordering index scan hand its ORDER BY value to the target list