Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

On Fri, Aug 08, 2025 at 09:09:20AM -0700, Nikhil Kumar Veldanda wrote:
> I have a question regarding TOAST pointer handling.
>
> As I understand, in the current design, each attribute in a HeapTuple
> can have its own TOAST pointer, and TOAST pointers are possible only
> for top-level attributes.
>
> Would it make sense to maintain an array for ttc_toast_pointer_size in
> ToastTupleContext, allowing us to estimate the size per attribute
> based on compression or other criteria?
>
> This approach could make the logic more generic in my opinion, but it
> would require changes in toast_tuple_find_biggest_attribute and other
> places.
>
> I’d like to hear your thoughts on this.

Yes, that's some complexity that you would need if plugging in more
vartags if these are related to compression methods, and also
something that we may need if we use multiple vartags depending on a
value ID assigned (aka for the TOAST table with 8-byte values, short
Datum if we have a value less than 4 billion with one vartag, longer
Datum if value more than 4 billion with a second vartag).

For the 4-byte vs 8-byte value case, I was wondering if we should be
simpler and less optimistic and assume that we are only going to use
the wider one depending on the type of chunk_id in the TOAST table, as
a minimum threshold when checking if a tuple should be toasted or not.
Perhaps my vision of things is too simple, but I cannot think about a
good reason that would justify making this code more complicated than
it already is.
--
Michael