Re: BUG #19519: REPACK can fail due to missing chunk for toast value

On Wed, 5 Aug 2026 at 20:18, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>
>
>
> > On 31 Jul 2026, at 00:54, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com> wrote:
> >
> > <v5-0002-Various-fixes-and-adjustments.patch>
>
[...]
> So v5 cannot be backpatched as
> is without breaking external table AMs. As I understand that the current
> plan is to settle the fix for HEAD first and consider a separate
> ABI-preserving back-branch variant afterwards.

Exactly.

> Three issues identified in the thread remain unresolved in the posted v5
> patch set:
>
> * As Zhijie reported, TOAST_MISSING_OK is lost for tuples stored in
> rs_unresolved_tups, including those inserted by end_heap_rewrite().

Fixed in the attached patch. I've adjusted various signatures with
recently_dead instead of toastflags, as toast flags don't make much
sense to pass around in components that deal with tuples that may or
may not contain externally toasted data.

> * As Matthias noted, the scan-and-sort path still has a check/use race
> because it discards the detoasted values before putting the original
> tuple into tuplesort.

Fixed in the attached patch, by rebuilding the heap tuple from the
detoasted values, and inserting that "fat" heap tuple into the
tuplesort.

> * As Dilip noted, the TOAST_MISSING_OK path for a TYPSTORAGE_PLAIN
> attribute uses detoast_external_attr_extended() instead of
> detoast_attr(). On a successful fetch, an external compressed datum
> therefore appears to remain compressed.

I _think_ I've fixed that in the attached version.

> I think the memory-leak fix in v5-0002 misses one exit path. Detoasting
> now precedes the partial index predicate, but if ExecQual() returns false,
> the loop continues without freeing detoasted_attrs and the detoasted
> values.

Thanks for the report, fixed.

> I also have a question about the concern Ekaterina raised in the related
> thread regarding real corruption. With TOAST_MISSING_OK, v5-0002 also
> returns false for an existing chunk with an unexpected size or a chunk
> outside the requested range. Is concurrent removal expected to produce
> those cases too, or could the tolerant path be limited to a missing chunk
> or a gap?

I think it is possible for chunks to be missed, as this can be caused
just by pruning (the tuple will be set LP_DEAD, and thus skipped by
the SysScan). Unexpected size or chunks outside the expected range are
also possible, but those would have to be caused by a toast-table-only
VACUUM cycle (and subsequent reuse of the OID) or reuse of cached
indexscan results after the TIDs are reused.

Attached is a flattened v6, containing v4 + the fixes descibed above.

Kind regards,

Matthias van de Meent
Databricks (https://www.databricks.com)