Re: Tepid: selective index updates for heap relations

Hey Peter,

In v79 I said 0001-0008 could be reviewed without the rest. Nobody has
commented on them (yet), which I put down to their sitting in front of
eleven heap patches and two test-only ones. So here is what they cover
and who needs them.

In 2021 you asked that we "have a generalized definition that accounts
for what variation is acceptable, and (most importantly) what variation
*isn't* acceptable" [1], and Tom said we'd "need some interlocks on
which index AMs could be used with which table AMs", and that
tidbitmap.c is "quite specific to heap-style TIDs" [2]. amgetbatch is
about to make the index-to-table interface new again [3]. 0001 is my
answer to that for the row locator. A table AM returns a LocatorDesc:

width exact size, or a bound when negative; index builds
refuse anything wider than ItemPointerData unless the
index AM opts in (0005)
max_offset the largest offset; bounds bitmap scans and GIN
(Jeff's 2021 problem)
stable does UPDATE keep the row's locator?
old_version_retained
can the pre-update version still be fetched by locator?

amlocator.h also writes down what every locator must satisfy, which
no field relaxes: it travels as an ItemPointerData, it is the row's
ctid, certain values are reserved, and locators sort in (block,
offset) order, which nbtree, ANALYZE and BRIN depend on. Peter's
2021 view was that TIDs should stay fixed size and correlate with
physical layout [4]. 0001 keeps that as the rule. 0005 is an opt-in
for an out-of-core index AM and table AM pair that agree on something
wider. No in-core AM uses it.

0002-0004 change interfaces that carried heap's answers. 0005-0008
are gated on the descriptor. Heap's behavior is unchanged, and with
all eight applied every case in v79's benchmark was within 1.2% of
master.

0002-0003 TM_FailureData without ctid/xmax/cmax; "traversed" becomes
"retargeted", which is what callers act on
0004 flags for when the executor names a version it has already
locked: after a BEFORE ROW trigger or EvalPlanQual, in ON
CONFLICT, MERGE and logical replication apply
0005 amcanvarlocator
0006-0007 RETURNING OLD, AFTER ROW triggers, transition tables and
EvalPlanQual stop re-fetching old rows by locator when
old_version_retained is false
0008 per-group BitmapAnd/BitmapOr inexactness callbacks, for
Tom's tidbitmap point; 0015 uses them for Alexander's
false negative

FLUX (CF 7389) is an in-place table AM, and it shows the cost of not
having this. It carries its own versions of 0006-0007 behind a single
am_inplace_update_keeps_tid flag. It also needs 0004. Without it FLUX
relaxes two assertions and infers "already locked" from per-command
bookkeeping and which lock flags the caller passed. That inference has
already produced a MERGE loop and an ON CONFLICT crash that FLUX had to
fix one at a time. I'd rather FLUX used this contract than add a second
set of hooks for the same things. The test module in 0020
(test_locator_tableam, not for commit) runs heap variants that declare
different width, max_offset, old_version_retained and bitmap_or_inexact
values. stable has no reader in core yet; it's there for the lock and
update callbacks' documented contract. If reviewers would rather it
wait for a consumer, I'll drop it.

If it helps, I'll post 0001-0008 as their own thread and CF entry,
rebased on master without the heap patches, and keep this thread for
0009-0019.

Objections? Thoughts? Thoughts? Thoughts? Thoughts?

-greg

[1] https://postgr.es/m/CAH2-Wzmee=VKQ7DbgZY4AcacWSoWTw0yyLdnvtn6kyH0nzg+pw@mail.gmail.com
[2] https://postgr.es/m/3268871.1619800534@sss.pgh.pa.us
[3] https://postgr.es/m/auOon54DuG2A6ll4%40floki
[4] https://postgr.es/m/CAH2-Wzk6FHes6H-cCN=1pxoVauza4Rm5c0RC1dyB6qBfdPPYDQ@mail.gmail.com