Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

Hi michael,

I have a question regarding TOAST pointer handling.

As I understand, in the current design, each attribute in a HeapTuple
can have its own TOAST pointer, and TOAST pointers are possible only
for top-level attributes.

Would it make sense to maintain an array for ttc_toast_pointer_size in
ToastTupleContext, allowing us to estimate the size per attribute
based on compression or other criteria?

This approach could make the logic more generic in my opinion, but it
would require changes in toast_tuple_find_biggest_attribute and other
places.

I’d like to hear your thoughts on this.

On Fri, Aug 8, 2025 at 12:52 AM Michael Paquier <michael(at)paquier(dot)xyz> wrote:
>
> On Fri, Aug 01, 2025 at 06:03:11PM +0900, Michael Paquier wrote:
> > Please find attached a v3, that I have spent some time polishing to
> > fix the value ID problem of this thread. v2 had some conflicts, and
> > the CI previously failed with warning job (CI is green here now).
>
> Attached is a v4, due to conflicts mainly caused by the recent changes
> in varatt.h done by e035863c9a04. This had an interesting side
> benefit when rebasing, where I have been able to isolate most of the
> knowledge related to the struct varatt_external (well
> varatt_external_oid in the patch set) into toast_external.c, at the
> exception of VARTAG_SIZE. That's done in a separate patch, numbered
> 0006.
>
> The rest of the patch set has a couple of adjustements to document
> better the new API expectations for toast_external.{c,h}, comment
> adjustments, some more beautification changes, some indentation
> applied, etc.
>
> > As things stand, I am getting pretty happy with the patch set up to
> > 0005 and how things are getting in shape for the interface, and I am
> > planning to begin applying this stuff up to 0005 in the next couple of
> > weeks.
>
> As of this patch set, this means a new target of 0006, to get the
> TOAST code refactored so as it is able to support more than 1 type of
> external on-disk pointer with the 8-byte value problem in scope. Any
> comments?
> --
> Michael

--
Nikhil Veldanda