Re: [PATCH] Halve peak memory allocation for toast_flatten_tuple

Hi Pavel,

I could not find measurements in the thread, so I made some. Two kinds:
the peak inside toast_flatten_tuple() itself, and the peak RSS of the
whole backend.

Setup: one row with REPLICA IDENTITY FULL, columns left at the default
EXTENDED storage and filled with semi-compressible text so the values are
stored out of line and compressed. wal_level = logical, workload
UPDATE t SET id = id + 1. Run on master (068c9aaec3) and with v1 applied.
The plain payload is held constant and split across a varying number of
columns. Note toast_flatten_tuple() calls detoast_external_attr(), which
fetches the value but leaves it compressed, so the memory at stake is the
stored size, not the plain size.

For the in-function figures I instrumented both versions to record
MemoryContextMemAllocated() across the call: in master right after
heap_form_tuple(), where every detoasted value is still live, and in v1
inside the per-attribute loop, where the tuple and one detoasted value are
live. The backend figures are simply VmHWM read before and after the
UPDATE.

Peak memory allocated inside toast_flatten_tuple():

cols plain stored master patched cut
---- -------- --------- ---------- ---------- ------
1 96 MB 28 MB 57 MB 57 MB 0.0%
2 96 MB 28 MB 57 MB 43 MB 25.0%
4 96 MB 28 MB 57 MB 36 MB 37.5%
8 96 MB 28 MB 57 MB 32 MB 43.7%
8 896 MB 265 MB 530 MB 298 MB 43.7%

That is your claim confirmed, to three decimals: master is exactly 2x the
tuple in all five cases, and v1 is exactly 1x plus the one detoasted value
it is holding.

Whole-backend peak RSS:

cols plain stored master patched cut
---- -------- --------- ---------- ---------- ------
1 96 MB 28 MB 89 MB 88 MB 0.4%
2 96 MB 28 MB 88 MB 83 MB 6.4%
4 96 MB 28 MB 89 MB 75 MB 15.0%
8 96 MB 28 MB 88 MB 72 MB 18.8%
8 896 MB 265 MB 803 MB 580 MB 27.8%

One small thing on the code, since I was reading it anyway.

toast_flatten_tuple() now open-codes what heap_form_tuple() does: the
hasnull scan, len and hoff, heap_compute_data_size(), the palloc0 and the
HeapTupleHeaderSet* calls. It has to, because it needs the allocation and
the fill separated.

You already split fill_val() and heap_fill_tuple() into _attr variants with
thin wrappers. The same move one level up would avoid the duplication
in toast_flatten_tuple():
heap_form_tuple() becomes a wrapper around heap_form_tuple_attr(), which
takes skip_copy and attr_data_out. toast_flatten_tuple() could then call
that and drop the block entirely.

Worth it for the single source of truth, I think.

Regards,
Johannes Edmeier

On Thu, 15 Oct 2026 at 15:16, Pavel Borisov <pashkin(dot)elfe(at)gmail(dot)com> wrote:
>
> Hi, hackers!
>
> Existing code detoasts every external attribute into a Datum array,
> then calls heap_fill_tuple on the complete set. So around 2x tuple
> length memory is allocated (result tuple plus all detoasted values).
> If I remember correctly, toasted tuples can be up to 1GB long, so
> even before the length limit, tuple flattening can strain the local
> backend's memory beyond the need.
>
> Mainly this affects logical replication with REPLICA IDENTITY
> FULL with tables with several large toasted columns: every
> UPDATE/DELETE flattens the old tuple. The other callers -- catalog
> cache loading and PL/pgSQL record assignment -- handle smaller tuples
> where the difference is negligible.
>
> In the patch attached I flatten TOASTed tuples with a two-stage approach:
> - For each external attribute build a VARHDRSZ stub whose header
> matches what detoast_external_attr will return (size and compression
> tag). Pass the stubs through heap_fill_tuple_attr with skip_copy=true
> so alignment and sizes are computed without copying data. attr_data[]
> records the destination pointer for every non-null attribute.
> - Detoast one attribute at a time into the pre-computed position, pfree
> immediately.
>
> Peak memory usage decreased from 2x tuple to 1x tuple plus one detoasted
> value.
>
> The extra per-attribute cost (one VARHDRSZ palloc/pfree for the
> stub, toast pointer decode for compression detection) is negligible
> compared the toast-fetch I/O that dominates every call.
>
> Please look at the patch attached, reviews are always welcome!
>
> Regards,
> Pavel Borisov
> Supabase

--
JOHANNES EDMEIER
Co-Founder & Head of Engineering
steadybit GmbH

johannes(dot)edmeier(at)steadybit(dot)com
+49 (0) 151 1086 7077

steadybit.com
Hochstrasse 11 | 42697 Solingen | Germany
Commercial Register: District Court Wuppertal | HRB 30206 | Germany