zstd TOAST: reusing the decompression context, on v7 -- evidence (2026-10-15) Tree: master 54635e6892f + v7-0001..0003 (the base Greg used). zstd 1.5.7. 1. Integrated 0003 v7-0003-con-dctx-reuse.patch = Nikhil's v7-0003 with the reuse folded in; author unchanged. Against his 0003: toast_compression.c only, +34/-14 (v7-reuse-zstd-dctx-sobre-v7.diff), plus one paragraph in the commit message. - one static ZSTD_DCtx per backend, created on first use, ZSTD_DCtx_reset(session_only) at the start of each use; full reads use ZSTD_decompressDCtx(), slices the existing streaming loop - the slice no longer frees the context; the stale comment about not leaking it is gone git am of v7-0001 + v7-0002 + this 0003 on 54635e6892f gives the tree that was built and tested. Builds with 0 warnings; make check: all 240 tests pass (-O2, and cassert + ASan). 2. Speed (paired, hot cache, bench.sh as on 26-09; ratio v7+reuse / v7; * = every round faster) 32 kB slice/prefix docs 0.71/0.72* json 0.62/0.66* random 0.72/0.75* (v4: 0.66-0.77) 32 kB full read docs 0.87* random 0.86* json 0.99 (v4: 0.87-0.89) 1 MB random slice 0.67*; other 1 MB points 0.98-1.01 (6 rounds) 4 kB 0.92-0.98; load: unchanged (the change does not touch compression) Tables: zstd-dctx-reuse-paired-v7.txt, zstd-dctx-reuse-paired-v7-1m.txt. 3. Correctness of the reused context (check_dctx.sh, corrupt_block.py) A 1 MB zstd value (8 blocks) is corrupted on disk: its third block header gets the reserved block type, so decoding fails there, mid-frame, after two good blocks. Then, in one backend: corrupt slice -> ERROR, good slice and full read right after -> correct; corrupt full read -> ERROR, then good full read and slice -> correct; prefix of the corrupt value (before the bad block) -> correct; an early-stopping prefix then a slice and a full read of another value -> correct; the error inside a savepoint, rolled back, then a good slice -> correct; all 221 good values, slice and full, in one query -> correct. Each value checked by md5. v7 (context per value) pass v7 + reuse pass v7 + reuse, cassert + ASan pass, no ASan report v7 + reuse WITHOUT the reset FAIL: every slice after a failed slice errors, and after an early-stopping prefix the slice of another value returns WRONG DATA without any error. Full reads stay correct (ZSTD_decompressDCtx starts a new frame anyway). So the reset is what makes reuse safe, and the test detects its absence. Error paths: no PostgreSQL function is called while zstd holds the context (the ERRORs come after ZSTD_decompressDCtx() / the streaming loop return), so no longjmp can leave it mid-call; if ZSTD_createDCtx() fails the static stays NULL and the next use retries. The context is never freed, so there is nothing to free twice or use after free. ASan intercepts zstd's malloc/free (libzstd itself is not instrumented). 4. Memory held by the context (mem_dctx.c, ZSTD_sizeof_DCtx, used as the TOAST code uses it) new, or after full reads only (one-shot API) 96 kB after slices of 32 kB values 160 kB after slices of 1 MB values 1.2 MB after slices of 8 MB or 64 MB values 2.5 MiB (window capped at 2 MiB, level 3) then 128 slices of 32 kB values 160 kB (zstd shrinks oversized buffers) v7 allocates the same buffer on every slice and frees it; the reuse keeps at most ~2.5 MiB per backend that has sliced a large zstd value, until zstd drops it. malloc'd by zstd, outside memory contexts: resets and aborts do not touch it, which is what the checks in 3 rely on. A future compression-level setting would raise the cap with the window. 5. Scope vs the WAL precedent 857340b0263 keeps a static ZSTD_CCtx for the life of the backend; ce3387b74b7 ties the ZSTD_DCtx to the XLogReaderState, the object that owns WAL decoding. Decompression of TOAST values has no such owner (detoasting is called from everywhere), so the backend is the natural scope, as for the WAL compression context; the reset at each use gives the isolation, and the cost is bounded by 4. No shared zstd helper exists in master; each subsystem keeps its own context. 6. Greg's v20261015-0004 (dictionaries) v20261015-0001..0003 are v7 with shifted hunk offsets only. 0004 applies unchanged on top of the integrated 0003. Its dict_dctx serves only the dictionary method (zstd_dict_decompress_datum*, toast_dictionary.c) and only through the one-shot API; ours serves the plain zstd method in toast_compression.c. Different statics, different call paths: no shared state. If 0004 later moves its slice to the streaming loop with ZSTD_DCtx_refDDict(), as its own comment suggests, it will need the same reset at each use (3 shows what happens without it). 7. Not done: the end-to-end pgbench (pgbench_v7.sh) did not run; the machine was busy.