Re: Support for 8-byte TOAST values, round two

On Mon, Aug 10, 2026 at 09:37:43AM +0200, Hannu Krosing wrote:
> Are function pointers really that much of a concern considering that
> we use tables of virtual pointers in core code paths like Table and
> Index Access Methods?

I am pretty sure that TOAST paths could get hotter than AMs under some
COPY-related workloads.

> Was this performance impact actually measured ?

I didn't measure any. Another comment I had was "overengineered",
which is perhaps fair as the TOAST code is quite linear in terms of
internals.

> Do I understand correctly that with this patch the selection between
> oid and oid8 happens at the table level not at the individual toasted
> item level ?

Yes, it's a design choice to use a table-level comparison. Choose one
with the reloption, then stick to it. Across upgrades, we keep the
same TOAST table. Rewrites are the same: no changes after a VACUUM
FULL, same atttype. There is nothing difference than what we do now,
which is why it's appealing to me.

> Maybe it would make sense to get better backwards compatibility and
> smoother upgrades by doing the same I did in my direct toast patch to
> allow both versions to live in the same table, namely to add a
> separate oid8 column and use conditional indexes for both new oid8 and
> old plain oid lookups?

Backward-compatibility and upgrades are handled in the patch set.
And TBH, I don't like much mixing multiple vartags in a single TOAST
table because it makes the whole layer more annoying to deal with.
Sticking with one atttype for each toast table is simpler. One
optimization would be to allow a OID external pointer to point to a
oid8 TOAST table, which offers benefits as long as we have less than 4
billion OIDs generated. This would not last long for some
deployments. Another idea that I found overkill was the use of a
per-toast sequence to generate the numbers. This can also be built on
top of what I have here, just found that expensive and a single oid8
counter in the control file is fine for all tables in a cluster
anyway.

> In case of oid8 it would be
> * add a column for new chunk_id8 oid8
> * add a conditional index on the (chunk_id8, chunk_seq) WHERE
> chunk_id_oid8 IS NOT NULL
> * making the original toast index conditional on WHERE chunk_id IS NOT NULL

Err. You're making TOAST tables larger than they should with
attributes that may not be required because some attributes may not be
required. With the TOAST data, it does not matter overall, but I'm
not much a fan of adding more data to disk than necessary, with more
indexes to maintain on each insert.

As a whole I'm not planning to go beyond what's presented on this
thread. Sticking to the simpler plan of having 8-byte values would
make a lot of customers I know of quite happy, because we have no exit
route now when it comes to core. (Spoiler: if I commit a patch, I own
its maintenance, and I'm well.. OK with what I have sent with a
long-term maintenance picture in mind.)
--
Michael