Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

On Mon, Jul 21, 2025 at 02:06:03PM +0300, Nikita Malakhov wrote:
> While it is very fast on small data - I see several disadvantages:
> - first of all, VACUUM should be revised to work with such tables;
> - problematic batch insertion due to necessity to store TID chain.
>
> It is just a POC implementation, so please don't blame me for
> questionable decisions.
>
> Any opinions and feedback welcome!

I think that this is going to be helpful in taking some decisions with
the refactoring pieces I am proposing for the external TOAST pointer
layer. You have some interesting points around
detoast_external_attr() and detoast_attr_slice(), as far as I can see.
One point of the on-disk TOAST refactoring is that we should be able
to entirely avoid this level of redirection. I get that this is a
POC, of course, but it provides pointers that what I've done may not
be sufficient in terms of extensibility so that seems worth digging
into.

The patch posted by Nikhil is also something that touches this area,
that I have on my tablets:
https://www.postgresql.org/message-id/CAFAfj_E-QLiUq--%2BKdyvb%2B-Gg79LLayZRcH8%2BmFPzVuDQOVaAw%40mail.gmail.com

It touches a different point: we need to be smarter with
CompressionToastId and not use that as the value for what we store on
disk. Each vartag_external should be able to use the compression
methods values it wishes, with the toast_external layer being in
charge of translating that back-and-forth with the GUC value.
--
Michael