Re: ZSTD TOAST compression, and an extensible compression method encoding

Hi Greg,

On Mon, Nov 2, 2026 at 7:46 PM Greg Burd <greg(at)burd(dot)me> wrote:
> Manu, the v13 containment fixes are unchanged in v14, so your repros
> for them should still hold. The new parts are the header validation
> in 0002 and the UPDATE fast path in 0004, if you want to point your
> harness at them.

They hold: v14 gives the same results as v13, and 4(a) still fails
without Muzzammil's v1 and passes with it.

Header validation: of 1044 corrupt inline headers (4 to 12 bytes, both
forms, va_cmid 0-255), with five readers each under cassert+ASan, none
reads past the datum or fails an assertion. Hannu's toast_save_datum()
point reproduces: va_cmid 3 or >= 6 fails its Assert through UPDATE,
INSERT ... SELECT and CTAS, and his change makes all of them report
"compressed data is corrupt".

UPDATE fast path: 1M unrelated-column UPDATEs of values on a
non-current dictionary stay HOT and take 0.9 s and 128 MB of WAL, where
v13 takes 5.4-5.7 s and 460 MB; VACUUM FULL, CLUSTER, REPACK and REPACK
(CONCURRENTLY) then recompress them and retire the old dictionary.

Among 141 uses of the TOAST predicates and 24 other container paths,
Hannu's check_domain_for_new_field() case is the only one missing; his
one-line change fixes it in the repro. The tables are attached.

Manu