Re: ExecForceStoreHeapTuple() loses tts_tid, so ORDER BY-op index scans project an invalid ctid

On Tue, Sep 8, 2026, at 8:03 PM, Michael Paquier wrote:
> On Tue, Sep 08, 2026 at 01:28:22PM -0400, Greg Burd wrote:
>> ExecForceStoreHeapTuple() does not set slot->tts_tid when the target
>> slot is a TTS_IS_BUFFERTUPLE slot. Any plan that re-stores a heap tuple
>> through it and then projects ctid therefore gets (4294967295,0) instead
>> of the row's real heap TID.
>
> Oops.

Yeah, oops indeed. :)

> My question would be what kind of testing you have done to spot that..

Honestly, not by a test. I am working on a new index AM for vector
similarity search [1] adding a 1-bit encoding with rerank. To do that
rerank it sets xs_recheckorderby = true, because its distances are
quantised and only the executor's exact re-check can order the top-k
correctly. That routes tuples through nodeIndexscan.c's reorder queue.
One of our documented recipes harvests ctid from a scan and chains it
downstream; that quietly started matching nothing, with the ctids
coming back as (4294967295,0).

My first assumption was that I'd broken something, so I went looking for
the boundary:

- xs_heaptid was correct and every real column was right; only the
projected ctid was wrong, which pointed at the slot, not the AM.
- ExecForceStoreHeapTuple's TTS_IS_BUFFERTUPLE branch calls
ExecClearTuple (hence ItemPointerSetInvalid on tts_tid) and never
restores it, while the sibling tts_heap_store_tuple does; and
slot_getsysattr answers SelfItemPointerAttributeNumber straight out of
tts_tid.
- Then I reproduced it with core GiST and nothing else loaded, which is
the version in the patch.

To confirm the fix I built stock 18.4 with only that hunk applied and
ran a test:
unpatched patched
ctid self-join, expect 5 1 5
UPDATE ... WHERE ctid, expect 5 1 5
sentinel ctids at LIMIT 50 49/50 0/50

The UPDATE line is the one that bothers me most. No error, it just
affects the wrong number of rows.

As for why the tree doesn't catch it, AFAICT nothing in core projects
ctid from an ORDER BY-op index scan. The GiST kNN tests check ordering
and results, which are fine here the row data is never wrong. That's
what v1 adds a regress case for, and I checked it fails
(ctid_matches = 1) without the hunk and passes (5) with it, so it gates
the fix rather than just recording current output.

One detail that probably explains the longevity, IndexNextWithReorder
only queues a tuple when the AM's advertised ORDER BY value doesn't
compare equal to the recomputed one. GiST's bounding-box distance is
sometimes exact, so some rows keep their real ctid. Note the 49 of 50
above rather than 50. My index AM can't usefully bound its distance and
advertises -inf, so every tuple goes through the queue and every one
shows the sentinel. Partial in core, total out here.

>> ExecClearTuple() reaches tts_buffer_heap_clear(), which does
>> ItemPointerSetInvalid(&slot->tts_tid). The tuple is then copied in, but
>> tts_tid is left invalid. The sibling path, ExecStoreHeapTuple() ->
>> tts_heap_store_tuple() — *does* slot->tts_tid = tuple->t_self, so this
>> reads as a plain asymmetry rather than an intentional choice.
>
> That's strange. Once thing that I can see why scanning this file is
> the same code pattern in tts_buffer_heap_copyslot(), where a slot is
> similarly cleared in a copy-paste fashion.

Agreed.

> Andres?
> --
> Michael

best, thanks for looking,

-greg

[1] https://codeberg.org/gregburd/pg_turbovec https://github.com/gburd/pg_turbovec