Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: "Burd, Greg" <greg(at)burd(dot)me>
Cc: Nikita Malakhov <hukutoc(at)gmail(dot)com>, Hannu Krosing <hannuk(at)google(dot)com>, Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)
Date: 2025-08-08 07:51:45
Message-ID: aJWska1_ZiXUIETX@paquier.xyz
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Aug 01, 2025 at 06:03:11PM +0900, Michael Paquier wrote:
> Please find attached a v3, that I have spent some time polishing to
> fix the value ID problem of this thread. v2 had some conflicts, and
> the CI previously failed with warning job (CI is green here now).

Attached is a v4, due to conflicts mainly caused by the recent changes
in varatt.h done by e035863c9a04. This had an interesting side
benefit when rebasing, where I have been able to isolate most of the
knowledge related to the struct varatt_external (well
varatt_external_oid in the patch set) into toast_external.c, at the
exception of VARTAG_SIZE. That's done in a separate patch, numbered
0006.

The rest of the patch set has a couple of adjustements to document
better the new API expectations for toast_external.{c,h}, comment
adjustments, some more beautification changes, some indentation
applied, etc.

> As things stand, I am getting pretty happy with the patch set up to
> 0005 and how things are getting in shape for the interface, and I am
> planning to begin applying this stuff up to 0005 in the next couple of
> weeks.

As of this patch set, this means a new target of 0006, to get the
TOAST code refactored so as it is able to support more than 1 type of
external on-disk pointer with the 8-byte value problem in scope. Any
comments?
--
Michael

Attachment Content-Type Size
v4-0001-Refactor-some-TOAST-value-ID-code-to-use-uint64-i.patch text/x-diff 16.0 KB
v4-0002-Minimize-footprint-of-TOAST_MAX_CHUNK_SIZE-in-hea.patch text/x-diff 2.9 KB
v4-0003-varatt_external-varatt_external_oid-and-VARTAG_ON.patch text/x-diff 12.5 KB
v4-0004-Refactor-external-TOAST-pointer-code-for-better-p.patch text/x-diff 44.2 KB
v4-0005-Move-static-inline-routines-of-varatt_external_oi.patch text/x-diff 4.7 KB
v4-0006-Introduce-new-callback-to-get-fresh-TOAST-values.patch text/x-diff 6.5 KB
v4-0007-Add-catcache-support-for-INT8OID.patch text/x-diff 1.3 KB
v4-0008-Add-GUC-default_toast_type.patch text/x-diff 5.4 KB
v4-0009-Introduce-global-64-bit-TOAST-ID-counter-in-contr.patch text/x-diff 21.0 KB
v4-0010-Switch-pg_column_toast_chunk_id-return-value-from.patch text/x-diff 2.3 KB
v4-0011-Add-support-for-bigint-TOAST-values.patch text/x-diff 13.7 KB
v4-0012-Add-tests-for-TOAST-relations-with-bigint-as-valu.patch text/x-diff 19.4 KB
v4-0013-Add-support-for-TOAST-table-types-in-pg_dump-and-.patch text/x-diff 21.3 KB
v4-0014-Add-new-vartag_external-for-8-byte-TOAST-values.patch text/x-diff 15.5 KB
v4-0015-amcheck-Add-test-cases-for-8-byte-TOAST-values.patch text/x-diff 3.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2025-08-08 08:06:16 Re: Enhance statistics reset functions to return reset timestamp
Previous Message Jelte Fennema-Nio 2025-08-08 07:23:44 Re: New commitfest app release on August 19th