Re: BUG #19519: REPACK can fail due to missing chunk for toast value

On Tue, 7 Jul 2026, 01:19 Michael Paquier, <michael(at)paquier(dot)xyz> wrote:

> On Mon, Jul 06, 2026 at 08:21:41PM +0200, Matthias van de Meent wrote:
> > So, in effect, this is an issue that's quite similar to the CIC/RIC
> > bug of PG14<14.4, except that
> >
> > 1.) in this case it's SnapshotAny instead of a
> > registered-but-invisible MVCC snapshot;
> > 2.) the issue in this case only happens when toast pointers were
> > cleaned up and then are dereferenced, rather than HOT chains updated
> > and reclaimed since the scan started; and finally
> > 3.) this issue dates back a very long time, likely since TOAST, but
> > defininitely since PG14 introduced the GlobalVisTestFor apis.
>
> You mean 042b584c7f7d, revert of d9d076222f5b. That rings a bell.
>

Indeed

> > So I think the only way to fix this is either
> > 1.) we update every maintenance process that makes use of SnapshotAny
> > and could access TOAST tables to handle errors caused by missing TOAST
> > data when the base relation's tuple is RECENTLY_DEAD, and treat those
> > as "tuple cleanup has already started, we just weren't aware of it
> > yet, so ignore this tuple" (as I mentioned in [1]), or
>
> That seems like an option worth exploring to me, but I doubt that
> we'll be able to get something that can be backpatched due to how
> invasive it is. I strongly suspect that it would require at least one
> table AM change to cope with the fact that we want to let the callers
> be OK with TOAST chunks missing in some contexts when grabbing a toast
> slice. That's perhaps for the best if we think about potential
> regressions, this is scary and old enough that we may still be OK by
> living without a backpatch.
>

I suspect that for indexing we're lucky, in that we have full control
internally over which tuple data get fed into IndexBuildCallback. We can
make sure only fully detoasted attributes get passed on, and can be
implemented with some (a lot) of TRY/CATCH work in or around
FormIndexDatum. That said, I don't think that's very nice, and I'm also a
bit concerned about leaking things (memory, buffers, ...) in such an
approach.

For REPACK/VACUUM FULL/CLUSTER, I think we can do something similar, given
that this code too has a central loop processing the tuple data, though I
suspect it'll need a bit more work than the reindex one.

> 2.) we set the xmin of the CI/R backend to the current OldestXmin for
> > the relation(s) we're currently processing, and so block all cleanup
> > of the relevant (toast) data for the duration of this statement or
> > whenever we check for a newer xmin for the relation (whilst ignoring
> > our current one).
>
> Err, I don't think that would be safe anyway? It's rather uncommon in
> practice in schemas, but we could trigger function calls, like index
> expressions, domains, whatever, that access data of other tables while
> processing one table with a SnapshotAny. The CIC revert of 14.4 was
> about such cases.
>

True, but in indexes that is forbidden: mislabeling of non-INMUTABLE
functions causes broken indexes every day, and we can't be expected to make
it work there. Similarly, for REPACK it shouldn't evaluate any
non-indexable expressions, right?

Either way, if we can register a snapshot in the REPACK/INDEX backend whose
xmin is (*correctly*) registered with the OldestNonRemovableTransactionId()
of the relation, then concurrent cleanup won't happen for toast, and we'd
also avoid dereferencing those dangling toast pointers. We won't need to
actually use that snapshot for any scans; it'll avoid the issue as long as
our backend advertises an xmin <= the OldestXmin used in maintenance to
hold back pruning/vacuum on the relation.

(we can adjust that xmin every once in a while with a newer
OldestNonRemovableTransactionId (excl. our own backend) as long as we also
adjust the OldestXmin cutoff for the SnapshotAny visibility tests, but I
don't suggest to backpatch that even newer system, it'd add significant
complications to an already complicated system)

- Matthias

ps. sorry for the formatting, I'm writing this on my phone.