Re: Support for 8-byte TOAST values, round two

On Mon, Aug 10, 2026 at 2:08 AM Michael Paquier <michael(at)paquier(dot)xyz> wrote:
>
> On Sun, Aug 09, 2026 at 11:53:23PM +0200, Hannu Krosing wrote:
> > I understand from some earlier messages that in one of the patches you
> > refactored the code to make it easier to add new VARLENA on disk
> > types.
>
> (Top-posting?)

Not this time, just thought this was the right thread to ask general
questions :)

And I have now written me an Chrome extension which warns me when I am
top-posting in GMail by mistake, so should happen less frequently now.

> > Where should I start looking for this?
>
> Quoting the top message of the thread:
> ```
> The main feedback of the previous thread is that the previous
> implementation with its callbacks for each vartag was not liked much,
> and their were concerns with pointer redirections and
> performance. This patch set uses what I am calling the "brutal"
> approach, relying on a vartag_external of a varlena or the atttype of
> the TOAST relation to decide which external toast pointer we should
> use. This uses no function pointers, and patches the code to deal
> with Oid or Oid8 TOAST values where it matters. So, this time,
> performance cannot really be an issue.
> ```

Thanks, that explains why I could not find it.

I was hoping to us see move to using virtual function tables like in
all other places where we provide extensibility.

> The refactoring worked around a set of callbacks that could be
> assigned to specific vartag_external, and one of the feedback that I
> got is that this required function pointers, which could impact
> performance for bulk-read or bulk-write of TOAST data.

Are function pointers really that much of a concern considering that
we use tables of virtual pointers in core code paths like Table and
Index Access Methods?

Was this performance impact actually measured ?

> This refactoring is gone as of round two, switching to direct if/elif
> block.

Do I understand correctly that with this patch the selection between
oid and oid8 happens at the table level not at the individual toasted
item level ?

Maybe it would make sense to get better backwards compatibility and
smoother upgrades by doing the same I did in my direct toast patch to
allow both versions to live in the same table, namely to add a
separate oid8 column and use conditional indexes for both new oid8 and
old plain oid lookups?

This allows selecting the toast type at the time of creating the
toasted value and also to move back and forth between different toast
types at any time.

In direct toast the support for this kind of backwards compatibility required
* adding the colum to store the tid array replacing the index for
multi-block toast
* making the original toast index conditional on WHERE chunk_id IS NOT
NULL (or WHERE chunk_id <> 0) so that it would not need to generate
unneccessary oids

In case of oid8 it would be
* add a column for new chunk_id8 oid8
* add a conditional index on the (chunk_id8, chunk_seq) WHERE
chunk_id_oid8 IS NOT NULL
* making the original toast index conditional on WHERE chunk_id IS NOT NULL

-----------
Hannu