Re: Improve hash aggregate spilling by writing only the needed columns

Hi Mario,

Applies to edbe7ffdc8a, builds clean and passes the regression tests
on macOS/arm64 (clang, no LLVM) and Linux/aarch64 (gcc 11, LLVM 15),
both with cassert.

There is one problem, found only on the LLVM build:

JIT crash with grouping sets
============================

The new hash_spill_colmap field is inserted before all_pergroups, but
FIELDNO_AGGSTATE_ALL_PERGROUPS is still 54, so JIT-compiled transition
code loads hash_pergroup instead. That happens to be the same pointer
for AGG_HASHED, but not for AGG_MIXED, and once a spill triggers the
nullcheck recompile the generated code indexes the wrong array.

CREATE TABLE w AS
SELECT i, i % 1000 AS g,
CASE WHEN i % 7 = 0 THEN NULL ELSE 'v' || (i % 97) END AS txt,
(i % 13)::numeric / 3 AS num
FROM generate_series(1, 200000) i;
ANALYZE w;

SET work_mem = '64kB';
SET hash_mem_multiplier = 1;
SET enable_sort = off;
SET jit_above_cost = 0;
SELECT g, txt, count(*), sum(num)
FROM w GROUP BY GROUPING SETS ((g), (txt));

server closed the connection unexpectedly

#0 evalexpr_0_10 ()
#1 ExecEvalExprNoReturn (executor.h:431)
#2 advance_aggregates (nodeAgg.c:825)
#3 agg_retrieve_direct (nodeAgg.c:2558)

Master with the same settings is fine, so is jit=off. count(*) alone
does not crash; any aggregate with a transition argument does. The
regression tests do not reach it because nothing there lowers
jit_above_cost.

Bumping the define to 55 fixes it here. See src/backend/jit/README,
"Type Synchronization", for the rule.

Other than that
===============

With the define fixed, forced-spill hash aggregation matches
enable_hashagg=off on ~30 query shapes without JIT and on 16 of
them with JIT (sparse columns, NULLs, by-ref and toasted values,
FILTER/HAVING-only columns, grouping sets, rescans, multi-level
respill).

One minor thing: hashagg_spill_tuple() and hashagg_spill_minimal_tuple()
both return total_written and both callers discard it; Disk Usage
comes from LogicalTapeSetBlocks(). Make them void?

I'll set the CF entry to Waiting on Author.

--
Xiaoxuan Li
Amazon Web Services