From f988bf572d6d9aec790bb73bcd311566dcf7400f Mon Sep 17 00:00:00 2001 From: Hannu Krosing Date: Sun, 23 Aug 2026 20:17:07 +0000 Subject: [PATCH v4 6/9] Add documentation for Direct TOAST Update the PostgreSQL SGML documentation to describe Direct TOAST: - doc/src/sgml/storage.sgml: Document Plain and Direct on-disk TOAST formats, TID-based direct addressing, single chunk, flat arrays, and hierarchical DAGs. - doc/src/sgml/config.sgml: Document the toast_flavour configuration parameter. - doc/src/sgml/ref/create_table.sgml: Document the toast_flavour storage parameter. --- doc/src/sgml/config.sgml | 19 ++++++++ doc/src/sgml/ref/create_table.sgml | 18 ++++++++ doc/src/sgml/storage.sgml | 70 ++++++++++++++++++++++++------ 3 files changed, 94 insertions(+), 13 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 0165eb9ec02..705c7d243b1 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -10508,6 +10508,25 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; + + toast_flavour (enum) + + toast_flavour configuration parameter + + + + + This variable sets the default TOAST storage flavour + to use for new writes. Supported values are plain + (the default, which uses index-based chunk lookups) and direct + (which stores physical TID pointers to bypass index lookups). + This can be overridden for individual tables by setting the + toast_flavour storage parameter in + CREATE TABLE or ALTER TABLE. + + + + temp_tablespaces (string) diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index fef24d8f3a2..a8f413098d3 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -1683,6 +1683,24 @@ WITH ( MODULUS numeric_literal, REM + + toast_flavour (enum) + + toast_flavour storage parameter + + + + + Sets the TOAST storage flavour for out-of-line data + written to this table. Supported values are plain (the + default, which uses index-based chunk lookup) and direct + (which stores physical TID pointers to bypass index lookup). + If not set, the default is determined by the configuration parameter + . + + + + parallel_workers (integer) diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index 83de016eaa5..bbc1b1a42c5 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -427,19 +427,63 @@ belonging to the owning table. Every chunk_id (an OID or an OID8 identifying the particular TOASTed value), chunk_seq (a sequence number for the chunk within its value), -and chunk_data (the actual data of the chunk). A unique index -on chunk_id and chunk_seq provides fast -retrieval of the values. A pointer datum representing an out-of-line on-disk -TOASTed value therefore needs to store the OID of the -TOAST table in which to look and the specific value -(its chunk_id). For convenience, pointer datums also store the -logical datum size (original uncompressed data length), physical stored size -(different if compression was applied), and the compression method used, if -any. Allowing for the varlena header bytes, -the total size of an on-disk TOAST pointer datum is 18 -bytes when using an OID as chunk_id, or 22 bytes -when using an OID8 as chunk_id, regardless of the -actual size of the represented value. +chunk_data (the actual data of the chunk), +chunk_tids (an array of chunk tuple identifiers, used by Direct TOAST), +and chunk_tid_offsets (an array of byte offsets, used for hierarchical Direct TOAST trees). + + + +PostgreSQL supports two flavours of on-disk TOAST +storage: + + + + Plain TOAST (the default): A unique index + on chunk_id and chunk_seq provides + retrieval of the values. A pointer datum representing a plain on-disk + TOASTed value stores the OID of the TOAST table + and the specific value (chunk_id). + Allowing for the varlena header bytes, the total size of a plain on-disk + TOAST pointer datum is 18 bytes when using an OID as + chunk_id, or 22 bytes when using an OID8 as + chunk_id, regardless of the actual size of the represented value. + + + + + Direct TOAST (toast_flavour = 'direct'): + Instead of looking up chunks through a B-Tree index, the TOAST + pointer directly stores the physical Tuple Identifier (TID) of the root/final chunk + in a 19-byte pointer datum. For single-chunk values (up to ~2 kB), reading the data + requires only a single buffer pin without any index access. For multi-chunk values up to 100 chunks, + the final chunk contains a chunk_tids array pointing directly + to all data chunks. For larger values (> 100 chunks), an internal hierarchical tree + is constructed with intermediate index nodes storing child TIDs and byte offsets in + chunk_tid_offsets, enabling efficient tree traversal and slice pruning. + In Direct TOAST, chunk_id is set to NULL, + and the unique index on the TOAST table is a partial index (WHERE chunk_id IS NOT NULL), + completely avoiding index maintenance and WAL logging overhead for direct chunks. + + + + + + +For convenience, pointer datums also store the logical datum size (original +uncompressed data length), physical stored size (different if compression was applied), +and the compression method used, if any. + + + +Direct TOAST is fully compatible with +pg_upgrade (including mode). +Because pg_upgrade --link preserves the physical block +and offset layout of heap and TOAST relation files, +physical TID pointers remain valid across major version upgrades. +Furthermore, plain and direct TOAST +pointers can seamlessly coexist within the same table, allowing tables upgraded from +older PostgreSQL versions to be read transparently and +gradually adopt Direct TOAST for new writes. -- 2.55.0.1082.g2b9226bbc0-goog