Re: Significant performance issues with array_agg() + HashAggregate plans on Postgres 17

From: Jeff Davis <pgsql(at)j-davis(dot)com>
To: David Rowley <dgrowleyml(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Scott Carey <scott(dot)carey(at)algonomy(dot)com>, pgsql-performance(at)lists(dot)postgresql(dot)org
Subject: Re: Significant performance issues with array_agg() + HashAggregate plans on Postgres 17
Date: 2026-05-05 01:27:03
Message-ID: 8179bf4b80956ffc983d48e127980ea84b0fe0da.camel@j-davis.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-performance

On Mon, 2026-04-20 at 16:40 +1200, David Rowley wrote:
> I was maybe wrong about just not bothering to handle
> MemoryContextSetParent(), but I'm not all that sure where the
> complexity is. Shouldn't it just be a matter of:
>
> If the context has a MemoryPool set, check if the parent has one too,
>    if not just swap parents out as the pool belongs to the context
> that's changing parent.
>    Else, gather memory totals for the swapping context and subtract
> from the MemoryPool, set the context being reparented's pool to NULL
> and change parent.
> else (no pool is set), just swap parent... I think.
>
> I think there might also need to be a check to see if the new parent
> has a pool and ERROR if it does. Maybe that's the messy part?

Patches attached.

I implemented everything, such that we don't need to ERROR.

It feels slightly over-engineered, but I just didn't like the idea of
erroring on what seem to be valid operations. Given the inheritance
behavior, you may not even be trying to use memory pools, and then
SetParent can still fail, and then what do you do?

Notes:

* It adds 3 extra fields to MemoryContextData inline. The out of line
approaches are not very clean: if we allocate in the context itself
reset will throw it away; if we allocate in the parent context then we
would need to move the allocation on SetParent(); allocating in the
caller means the caller needs to track it even though it has the same
lifetime; and I'm not sure it's a good idea to use malloc() directly.

* The "limit" terminology is a bit awkward because it doesn't really
enforce anything it just adjusts the max block size. Maybe there's a
better term for that?

* allocChunkLimit is not recalculated after SetParent(). I don't think
that's a correctness issue, but I might need to add some more comments.

I like the idea that memory contexts can inherit some information about
work_mem. I've wanted that to be possible for a while, and if we think
this is a good approach then we can expand it to other places in the
executor.

Regards,
Jeff Davis

Attachment Content-Type Size
v4-0001-Refactor-memory-accounting-into-inline-function.patch text/x-patch 8.4 KB
v4-0002-Memory-Pools.patch text/x-patch 14.4 KB
v4-0003-Update-Hash-Aggregation-to-use-memory-pools.patch text/x-patch 7.3 KB

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message Greg Sabino Mullane 2026-05-05 03:05:11 Re: Improving insert performance
Previous Message Laurenz Albe 2026-05-04 20:49:33 Re: Improving insert performance