| From: | Mario Karuza <mkaruza(dot)pg(at)icloud(dot)com> |
|---|---|
| To: | pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | tuplesort_putdatum() does not account for tuple memory |
| Date: | 2026-09-10 20:36:01 |
| Message-ID: | e9f9f54337cd86a982837763958518cdbf08863c.camel@icloud.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi hackers,
Memory allocated for copied pass-by-reference Datums was not accounted
against work_mem because tuplesort_putdatum() passed a hardcoded tuplen
of 0 to tuplesort_puttuple_common(). Function free_sort_tuple() adjusts
the accounting by the amount actually allocated, so freeing such a
tuple subtracts an amount that was never added.
This was introduced in 6ed83d5fa55, which switched non-bounded sorts to
bump contexts. That commit correctly changed the other tuplesort_put*()
functions to compute the size, leaving only this one passing hardcoded
0.
So currently:
1) Bounded datum sorts are misreported. With work_mem = 4MB:
EXPLAIN ANALYZE SELECT md5(i::text) AS hash
FROM generate_series(1,100000) i
ORDER BY hash LIMIT 5;
master: Sort Method: still in progress Memory: 0kB
patched: Sort Method: top-N heapsort Memory: 25kB
2) work_mem is not enforced against the tuple data, and hold more data
than allowed before spilling With work_mem = 4MB:
EXPLAIN ANALYZE SELECT md5(i::text) AS hash
FROM generate_series(1,100000) i
ORDER BY hash;
master: Sort Method: quicksort Memory: 3073kB
patched: Sort Method: external merge Disk: 3920kB
The attached patch computes tuplen the way the tuplesort_put*()
variants do.
Thanks,
Mario
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-Fix-memory-accounting-for-datum-sorts-of-pass-by-.patch | text/x-patch | 2.3 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2026-09-10 20:37:15 | Re: Trying to break online checksums with LLMs |
| Previous Message | Masahiko Sawada | 2026-09-10 20:34:27 | Re: Misplaced comment in snapbuild.c |