Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Burd, Greg" <greg(at)burd(dot)me>, Nikita Malakhov <hukutoc(at)gmail(dot)com>, Hannu Krosing <hannuk(at)google(dot)com>, Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)
Date: 2025-11-26 04:54:03
Message-ID: aSaH68oCJvAG8mwA@paquier.xyz
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Sep 30, 2025 at 03:26:14PM +0900, Michael Paquier wrote:
> There were a few conflicts, so here is a rebased v7, moving the patch
> to the next CF. I have been sitting on this patch for six weeks for
> the moment.

Attached is a rebased v8, fixing a couple of conflicts.

> Tom, you are registered as a reviewer of the patch. The point of
> contention of the patch, where I see there is no consensus yet, is if
> my approach of using a redirection for the external TOAST pointers
> with a new layer to facilitate the addition of more vartags (aka the
> 64b value vartag proposed here, concept that could also apply to
> compression methods later on) is acceptable. Moving to a different
> approach, like the "brutal" one I am naming upthread where the
> redirection layer is replaced by changes in all the code paths that
> need to be touched, would be of course cheaper at runtime as there
> would be no more redirection, but the maintenance would be a nightmare
> the more vartags we add, and I have some plans for more of these.
> Doing the switch would be a few hours work, so that would not be a big
> deal, I guess. The important part is an agreement about the approach,
> IMO.

This point still got no reply. It would be nice to do something for
this release regarding this old issue, IMO..
--
Michael

Attachment Content-Type Size
v8-0001-Implement-oid8-data-type.patch text/x-diff 38.7 KB
v8-0002-Refactor-some-TOAST-value-ID-code-to-use-Oid8-ins.patch text/x-diff 15.4 KB
v8-0003-Minimize-footprint-of-TOAST_MAX_CHUNK_SIZE-in-hea.patch text/x-diff 4.6 KB
v8-0004-Renames-around-varatt_external-varatt_external_oi.patch text/x-diff 20.3 KB
v8-0005-Refactor-external-TOAST-pointer-code-for-better-p.patch text/x-diff 46.3 KB
v8-0006-Move-static-inline-routines-of-varatt_external_oi.patch text/x-diff 4.7 KB
v8-0007-Split-VARATT_EXTERNAL_GET_POINTER-for-indirect-an.patch text/x-diff 5.9 KB
v8-0008-Switch-pg_column_toast_chunk_id-return-value-from.patch text/x-diff 3.6 KB
v8-0009-Add-catcache-support-for-OID8OID.patch text/x-diff 1.3 KB
v8-0010-Add-support-for-TOAST-chunk_id-type-in-binary-upg.patch text/x-diff 9.3 KB
v8-0011-Enlarge-OID-generation-to-8-bytes.patch text/x-diff 14.5 KB
v8-0012-Add-relation-option-toast_value_type.patch text/x-diff 6.5 KB
v8-0013-Add-support-for-oid8-TOAST-values.patch text/x-diff 16.9 KB
v8-0014-Add-tests-for-TOAST-relations-with-bigint-as-valu.patch text/x-diff 21.5 KB
v8-0015-Add-new-vartag_external-for-8-byte-TOAST-values.patch text/x-diff 14.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message shveta malik 2025-11-26 04:54:59 Re: How can end users know the cause of LR slot sync delays?
Previous Message Michael Paquier 2025-11-26 04:35:08 Re: change default default_toast_compression to lz4?