Re: Report planning memory in EXPLAIN ANALYZE

On Thu, 10 Aug 2023 at 03:12, Ashutosh Bapat
<ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> Thinking more about it, I think memory used is the only right metrics.
> It's an optimization in MemoryContext implementation that malloc'ed
> memory is not freed when it is returned using free().

I guess it depends on the problem you're trying to solve. I had
thought you were trying to do some work to reduce the memory used by
the planner, so I imagined you wanted this so you could track your
progress and also to help ensure we don't make too many mistakes in
the future which makes memory consumption worse again. For that, I
imagined you'd want to know how much memory is held to ransom by the
context with malloc(), not palloc(). Is it really useful to reduce the
palloc'd memory by the end of planning if it does not reduce the
malloc'd memory?

Another way we might go about reducing planner memory is to make
changes to the allocators themselves. For example, aset rounds up to
the next power of 2. If we decided to do something like add more
freelists to double the number so we could add a mid-way point
freelist between each power of 2, then we might find it reduces the
planner memory by, say 12.5%. If we just tracked what was consumed by
palloc() that would appear to save us nothing, but it might actually
save us several malloced blocks.

David