Re: Support for 8-byte TOAST values, round two

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
Cc: Greg Burd <greg(at)burd(dot)me>, Yugo Nagata <nagata(at)sraoss(dot)co(dot)jp>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Support for 8-byte TOAST values, round two
Date: 2026-09-14 05:00:49
Message-ID: aqd_gYs35v--sVlR@paquier.xyz
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Sep 11, 2026 at 08:44:26AM -0700, Bharath Rupireddy wrote:
> I reviewed v17 patches, overall they look good to me.
>
> v17-0001-Refactor-some-TOAST-value-ID-code-to-use-Oid8-in.patch: It
> v17-0002-Switch-pg_column_toast_chunk_id-return-value-fro.patch: It

I have discarded the change of reorderbuffer.c that was in v17-0001,
and added it back to what's now v18-0001. Then applied these two
after an extra round of self-review.

> v17-0003-Add-support-for-oid8-TOAST-values.patch: It looks good to me
> with one nit. Since chunk_id is always at attnum = 1 and all the
> toast_valueid_scankey_init() callers pass it as 1, and this init
> function is just to fetch the chunk_id, can we just hard-code it
> inside and remove the attnum function parameter?

If one wants to control which attribute number to look after, that
could be extended more later on, yes. I have removed the argument
from the function in v18-0001.

> v17-0004-Add-battery-of-tests-related-oid8.patch: It looks good to me.

Okay, thanks.

> pg_dump/pg_restore for demoing OID4 to OID8 migration for
> existing tables and TOAST table tests during pg_upgrade could be
> follow-up patches. This makes me think, if someone does an OID8 to
> OID4 migration and the chunk_ids are beyond the 4-billion limit, the
> restore should fail rather than silently wrapping the chunk_ids and
> causing TOAST index insert failures. My point is, we need to test this
> case as well.

Nah, I don't buy that this is worth the cycles. If one has the idea
to create an oid8 TOAST table, inserts more than 4 billion values into
it, dumps and restores into a table that uses a oid TOAST then the
result is an infinite loop in the backend on restored INSERTs.

Something that I still see as a hole is that we have the tables from
the strings test in the main regression test suite and we could
exploit that a bit more. Hmm. How about resetting the OID in the
control file earlier in the pg_upgrade test and check with a SELECT
post-upgrade that the counters generated by the main regression test
suite are more than 2^32?

Added also some tests for REPACK here in what's now 0002, based on the
feedback of upthread, to check that the a REPACK's rewrite does not
touch the TOAST table definition.

> v17-0005-Add-support-for-TOAST-pointers-as-oid8.patch: It looks good to me.

And rebased is a v18 with the three remaining pieces, plus two more
test patches: the tests of Rui with decoding and oid8 values higher
than 2^32, and my test to have the main regression test suite of
pg_upgrade run with a default oid8 in the control file higher than
2^32, checking that assigned chunk numbers are handled correctly.
--
Michael

Attachment Content-Type Size
v18-0001-Add-support-for-oid8-TOAST-values.patch text/plain 21.5 KB
v18-0002-Add-battery-of-tests-related-oid8.patch text/plain 38.3 KB
v18-0003-Add-support-for-TOAST-pointers-as-oid8.patch text/plain 46.7 KB
v18-0004-Decode-an-oid8-TOAST-value-with-an-ID-past-2-32.patch text/plain 2.5 KB
v18-0005-Add-check-for-2-32-oid8-chunk-values-across-the-.patch text/plain 2.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Richard Guo 2026-09-14 05:10:21 Re: remove_useless_joins vs. bug #19560
Previous Message Peter Smith 2026-09-14 04:40:14 Re: Review items for EXCEPT TABLE publication