Re: HashAggregate loses worker spill stats on rescan

Hi,

On Tue, 20 Oct 2026 at 01:49, Osama Abdul Qader
<osamaabdulqaderpostgres(at)gmail(dot)com> wrote:
>
> On Sun, 18 Oct 2026 at 22:55, Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com> wrote:
>>
>> Hi,
>>
>> HashAgg seems to lose the worker's spill stats when Gather is rescanned.
>> In the example below, Worker 0 spills on the first scan, but EXPLAIN shows
>> one batch and no disk usage after the second:
>>
>> CREATE TABLE hashagg_stats AS
>> SELECT value AS item, value % 10000 AS bucket
>> FROM generate_series(1, 1000000) AS value;
>> ALTER TABLE hashagg_stats SET (parallel_workers = 2);
>> ANALYZE hashagg_stats;
>>
>> SET max_parallel_workers_per_gather = 2;
>> SET max_parallel_workers = 1;
>> SET parallel_leader_participation = off;
>> SET parallel_setup_cost = 0;
>> SET parallel_tuple_cost = 0;
>> SET min_parallel_table_scan_size = 0;
>> SET work_mem = '64kB';
>> SET hash_mem_multiplier = 2;
>> SET enable_sort = off;
>>
>> EXPLAIN (ANALYZE, VERBOSE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
>> SELECT limits.maximum,
>> (SELECT count(*)
>> FROM (SELECT bucket, count(*) FROM hashagg_stats
>> WHERE item <= (SELECT limits.maximum)
>> GROUP BY bucket) AS grouped)
>> FROM (VALUES (1000000), (10)) AS limits(maximum);
>>
>> AFAICS, ExecEndAgg() clobbers the shared slot on each worker exit, rather
>> than combining the stats from successive scans.
>>
>> Patch attached to keep the memory/disk peaks and add the spill batches.
>> I think we want to count the initial batch only once, as in the
>> non-parallel case. Otherwise, wouldn't even a non-spilling rescan add
>> another batch?
>>
>> Regards,
>> Ayush
>
>
> Is the assumption that hash_batches_used is always at least 1 when
> ExecEndAgg() records the worker's statistics guaranteed by the
> surrounding code?

AFAICS yes for hashed/mixed aggs. ExecInitAgg() starts it at 1,
hashagg_spill_finish() increments it, and ExecReScanAgg() leaves it alone.

> Could the si->hash_batches_used == 0 check be affected by any rescan or
> worker-reuse scenario that would cause the initial batch to be counted
> more than once?

I don't see one so far. The slot is zeroed by ExecAggInitializeDSM(),
but not on a Gather rescan, so the next worker should just add to the
existing count.

Regards,
Ayush