notes.txt -- v21 of the zstd / TOAST dictionary series ===================================================== v21 applies on master as of 3ed10f282f9. 0004 changes as below, and 0002 has one comment rewrapped for pgindent; 0001, 0003 and 0005 have the same patch-ids as v20. 1. Values in PL/pgSQL record variables -------------------------------------- exec_move_row() fills a record variable with expanded_record_set_tuple(erh, tup, true, !estate->atomic). In a function (atomic) the last argument is false, so the tuple is copied without toast_flatten_tuple(). That is safe for TOAST pointers: a table cannot lose its TOAST data before the transaction ends. It is not safe for a dictionary-compressed value, whose dictionary row can go away inside the transaction: - a subtransaction that trained the dictionary rolls back (BEGIN ... EXCEPTION in an ordinary function is enough); - the table is rewritten after a newer dictionary was trained, which retires the old one in finish_heap_swap(); - the table is dropped. On v20 each of these, followed by reading the field, gave "compression dictionary N does not exist". Because GetNewOidWithIndex() can hand a deleted dictionary's OID to a new dictionary, the value could instead have been decoded with an unrelated dictionary. Fix: toast_flatten_tuple_dict() (heaptoast.c) expands only fields compressed with a dictionary and returns NULL when there are none; expanded_record_set_tuple() calls it whenever it would otherwise keep the tuple as is. It runs only for a tuple with HEAP_HASEXTERNAL set. toast_flatten_tuple(), toast_flatten_tuple_for_wal() and the new function share one workhorse selected by an enum, ToastFlattenMode, which replaces the keep_dict flag. Scalar variables were already safe: assign_simple_var() and the SELECT INTO path copy the value out of the tuple, decompressing it. Tests (compression_zstd_dict): cmrecrb (record and scalar variable, subtransaction rollback), cmrecfn('rewrite'), cmrecfn('drop'). Each fails on v20 and passes on v21. Checked on a cluster with no further change needed: a SQL function's result, a PL/Perl spi_exec_query() row and a PL/Python plpy.execute() row, each held across DROP TABLE, and a PL/pgSQL cursor. A deferred AFTER trigger and a FOR loop over the table cannot rewrite it ("pending trigger events", "being used by active queries"), so they do not reach this case. 2. Single-value tests --------------------- Every VARATT_IS_EXTERNAL() / VARATT_IS_EXTERNAL_NON_EXPANDED() test in src/backend, src/pl and contrib (33 places) was read and classified: a) Decides whether to make a value self-contained; now uses VARATT_IS_EXTERNAL_OR_DICT(): brin_tuple.c (brin_form_tuple), indextuple.c (index_form_tuple), tstoreReceiver.c, heaptoast.c (toast_build_flattened_tuple), expandedrecord.c (6 places). b) Tests one kind on purpose and handles dictionary values beside it: heaptoast.c (toast_flatten_tuple_internal, by mode), pl_exec.c (assign_simple_var, non-atomic path), datum.c (datumTransfer). c) Already decompresses any compressed value, which covers a dictionary value: heaptoast.c (toast_flatten_tuple_to_datum), fmgr.c (pg_detoast_datum_packed), varlena.c (text_substring), toast_helper.c:252, amcheck verify_nbtree.c. d) Not a decision about keeping a value: detoast.c internals, Asserts (toast_internals.c, reorderbuffer.c, ts_selfuncs.c), heaptuple.c fill_val (expanded and indirect flattening; the dictionary flag is set later in the same function), heapam.c HeapDetermineColumnsInfo, toast_helper.c toast_tuple_init (old pointer reuse; dictionary values go through toast_dictionary_keep() there), rangetypes.c datum_write (an error guard; range_serialize() detoasts first), amcheck verify_heapam.c and pageinspect (raw page reads), pgrepack.c repack_store_change. pgrepack.c spills only out-of-line attributes and writes the tuple as is to a file read back by the same REPACK (CONCURRENTLY). The old dictionaries are retired only in finish_heap_swap(), after the replay, so the spilled values stay readable. Checked: REPACK (CONCURRENTLY) of a table with two dictionaries while 40 rounds of UPDATE and INSERT ran left one dictionary, and all 2040 values read back correctly. The audit covers in-core code only; extensions are section 3. 3. Extension container types ---------------------------- A type whose values embed other varlenas, and which copies a value it is handed byte for byte, would carry a dictionary value out of its column. Core cannot see the value once it is inside the type's own bytes, so there is no place for a check. xtypes.sgml (TOAST Considerations) now says such a type must detoast what it embeds, and that PG_DETOAST_DATUM(), PG_DETOAST_DATUM_PACKED() and conversion through the output function decompress a dictionary value while a byte copy does not. toast_compression.h says the same next to VARATT_IS_COMPRESSED_DICT(). Survey, by reading the source: hstore PG_GETARG_TEXT_PP; hstore_from_record goes through the output function ltree, intarray, jsonb build from detoasted input pgvector DatumGetVector() is PG_DETOAST_DATUM() PostGIS PG_GETARG_GSERIALIZED_P() is PG_DETOAST_DATUM(); the header-slice path detoasts compressed values too None copies a varlena it was handed without detoasting it. 4. Cost for tables without dictionaries --------------------------------------- The v17 statement-level A/B on an m5d.metal (15 runs per cell in ABBA order, A/A noise floor 9%) found no measurable cost. To isolate fill_val(), v21 was measured with callgrind instruction counts, which do not depend on host noise. Release builds (-O3, no cassert) of the series base and of the series; one single-user backend per run; instructions counted only inside ExecutorRun of the INSERT: workload heap_fill_tuple Ir INSERT Ir 100k narrow rows, nothing in-line 27,740,006 -> 27,550,005 -0.08% compressed 20k rows, one in-line lz4 value 5,400,000 -> 5,500,000 +0.03% 20k rows recompressed to pglz 5,400,000 -> 5,500,000 +0.03% VARATT_IS_COMPRESSED_DICT() returns after a one-byte header test for a value that is not compressed in-line, so the narrow-row case pays nothing (the small drop is code layout). A compressed in-line value costs about five instructions. What I ran ---------- - Each patch on its own: -Werror build with no warnings, make check; pgindent and git diff --check clean. - test-world on 0001-0005: 396 passed, 36 skipped, 0 failed. - Builds with only lz4, only zstd, neither, and with ASan: make check passes, no sanitizer report. - The three cmrec tests fail with the 0004 of v20 and pass with v21.