Re: ZSTD TOAST compression, and an extensible compression method encoding

Hi Greg,

On Tue, Oct 27, 2026 at 11:57 AM Greg Burd <greg(at)burd(dot)me> wrote:
> Okay, v12 on master (065dc52f1a4).

v12 fixes all five on my side. It also applies cleanly on today's
master (76a55702d89), and make check (243) and amcheck pass with no
warnings with both libraries (-O2, cassert+ASan), lz4 only and zstd
only. A SIGINT on entry to the evaluation gives the usual ERROR, the
damaged chunk one WARNING and no "resource was not closed", and the
numbers match yours: 27,881 and 27,894 instructions per value with 2
fetches, and 7 CDicts per evaluation, 5 of them inside ZDICT.

The subtransaction also holds in VACUUM (PARALLEL 2, ANALYZE), BEGIN
READ ONLY, a DO block with EXCEPTION and autovacuum; an autovacuum
ANALYZE cancelled by LOCK TABLE inside the evaluation stops without
adopting, and a hot standby refuses ANALYZE and replays the adoption.

Containment also held everywhere I could take a value out of its
column before a rewrite retired its dictionary: INSERT ... SELECT,
CTAS, a whole-row composite, an array, another dictionary column, a
matview, partition row movement, a WITH HOLD cursor, a procedure
variable across COMMIT, logical decoding and an extended MCV list.
So did REPACK (CONCURRENTLY) with DML committed during the copy, and
pg_upgrade, which keeps the dictionary OIDs.

Two leftovers: the 0005 message still says compressed_size() and "any
failure is logged and swallowed, never aborting ANALYZE", and
toast_dictionary.h:39 still names compressed_size(); my diff missed it.

Thanks for taking all of these in.

Regards,
Manu