Re: BUG #19759: Hashed SetOp / RecursiveUnion pre-allocate up to 64 GB from the row estimate since 8f29467c57f

On Fri, 16 Oct 2026 at 02:47, PG Bug reporting form
<noreply(at)postgresql(dot)org> wrote:
> Since commit 8f29467c57f ("Change "long" numGroups fields to be Cardinality
> (i.e., double)", 2025-11-02), BuildTupleHashTable() no longer limits the
> initial size of a TupleHashTable to hash_mem. Its new comment says that
> staying within hash_mem is the caller's job. But nodeRecursiveunion.c and
> nodeSetOp.c pass the planner's raw group estimate, so the executor now
> allocates and zeroes a bucket array of up to 2^32 entries x 16 bytes =
> 64 GB in ExecInit, before reading a single row, whenever the estimate is
> large. PostgreSQL 18 is not affected (it capped nbuckets at
> get_hash_memory_limit() / entrysize). The commit is in REL_19_BETA1
> through REL_19_RC1.

Looking back at the changes made in 8f29467c5 and looking at the
estimation code in generate_recursion_path(), I think we're dreaming
if we think we can avoid capping planner estimates in the executor.
There's an argument for doing that in the planner, but plans can be
cached, and we don't replan when work_mem or the hash_mem_multiplier
changes, so I think keeping some logic in BuildTupleHashTable() is
sensible. Queries being slightly slower due to the hash table having
to grow a few times seems much better than OOM because of the planner
not having enough information to make an informed estimate.

I've tried to do that in the attached and incorporated the capping to
UINT32_MAX that Tom added in 8f29467c5. The calculation still doesn't
account for the tuple width, but if we were to try to include that as
part of the calculation is means making the cap calculation
susceptible to planner estimates. That seems weird since the whole
point of the cap is to protect against crazy planner estimates.

David