| From: | Tatsuya Kawata <kawatatatsuya0913(at)gmail(dot)com> |
|---|---|
| To: | Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Hash aggregation shows only the last execution's stats for parallel workers |
| Date: | 2026-11-16 14:16:12 |
| Message-ID: | CAHza6qfgOQ0Psd5kbYkHn2RCAOfcBMvRohUX3SUptd1kZj6euA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
I noticed that Agg has the same issue as the one fixed for Memoize [1].
When a Gather (or Gather Merge) node is rescanned, a new set of workers
is started, and they write their Agg stats into the same shared memory
slots as the previous ones. ExecEndAgg() overwrites the values, so
EXPLAIN ANALYZE shows the per-worker Batches etc. only for the last
execution.
=== Example
CREATE TABLE ag AS SELECT g % 2000 AS y FROM generate_series(1, 2000000) g;
ANALYZE ag;
SET work_mem = '64kB'; -- make the hash aggregation spill
SET parallel_leader_participation = off; -- only the worker runs the
Partial HashAggregate
SET max_parallel_workers = 1; -- so that Worker 0 runs both
executions
EXPLAIN (ANALYZE, VERBOSE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
SELECT o.z, (SELECT count(*) + o.z FROM (SELECT y, count(*) FROM ag GROUP
BY y) s)
FROM (VALUES (1), (2)) o(z);
=== Output on master
Each execution uses 5 batches (1 + 4), and Worker 0 runs the Partial
HashAggregate twice (loops=2), but Batches shows only 5:
-> Partial HashAggregate (actual rows=2000.00 loops=2)
...
Worker 0: actual rows=2000.00 loops=2
Batches: 5 Memory Usage: 161kB Disk Usage: 11048kB
=== With the patch
Batches: 9 Memory Usage: 161kB Disk Usage: 11048kB
The attached patch accumulates the values instead of overwriting them.
As for Memoize, it keeps the maximum for Memory Usage and Disk Usage.
For Batches, hash_batches_used is 1 when nothing spilled, so simply
adding the values would give 2 when a worker that didn't spill runs
twice, and Disk Usage would then be shown as if it had spilled. So the
in-memory batch is counted only once. This is the same as how master
counts it when the leader rescans the node itself.
Note that this example uses only one worker to keep the results stable,
and Worker 0 does the same work in both executions, so Memory Usage and
Disk Usage don't change with the patch. With two workers, the rows are
distributed differently in each execution, and Disk Usage then shows
the value of the last execution on master and the maximum with the
patch.
Regards,
Tatsuya Kawata
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-Accumulate-Agg-parallel-worker-stats-across-resca.patch | application/octet-stream | 1.8 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Joao Rito | 2026-11-16 14:16:50 | Re: [PATCH v2] Avoid carrying self lock-only xmax to updated tuple |
| Previous Message | Joao Rito | 2026-11-16 14:02:29 | Re: [PATCH] Check interrupts while draining received WAL |