Re: Subquery pull-up increases jointree search space

On 08/06/2026 17:27, Tomas Vondra wrote:
> On 6/8/26 15:32, Andrei Lepikhov wrote:
>> Instead, I used the heaptrack tool, which gave me the following
> Hmm, how does heaptrack deal with out memory pools? I was worried
> existing memory profilers (like heaptrack) would get confused by our
> memory contexts, attributing the whole block to the palloc that just
> happens to allocate a new block. But that's not really right.

In theory, heaptrack does not catch pallocs inside a memory context and only
tracks malloc and free calls. However, in my experience, it still helps identify
the issue. This might be because the planner uses local memory contexts, and
with [1] added, it could be even more localised. Usually, the problem is a cycle
that grows too long, with many allocations within it that go unreleased, causing
overconsumption and memory context growth. In this case, heaptrack may not point
to the exact spot, but it still provides useful hints.

Compared to other profilers (gperftools, jemalloc, valgrind massif), I like it
more for its flexibility, flamegraphs, and an interface that the support team
frequently uses (even on Windows somehow) - it is beneficial for the
inter-department communication. Profilers don’t understand malloc/palloc
postgres separation anyway, and to use them we need to annotate postgres calls.

>
> I see the heaptrack README claims it can work with memory pools after
> annotating the code in some way. But there's not much details about
> that. Also, it suggests valgrind/massif can already do that.
>

Looking into the interface [2,3], it looks quite clear to me - it even includes
a Valgrind macros set that makes it more natively integrated. Thinking about
implementation, my first wish was to create an extension and see how it actually
works in practice, but we can’t inject a wrapper into the static
MemoryContextMethods, which limits the ability of such an extension. Maybe with
an umbrella planning memory context …

Core-based annotation looks more interesting. There are questions about proper
palloc/malloc counting that exist - I think it can be generally solved by
employing heaptrack_pause / heaptrack_resume calls, if this topic sparks an
interest. Or we can discuss some API flexibility on the HeapTrack side.

[1]
https://www.postgresql.org/message-id/1f797d0e-4829-48b5-817e-2278466ed4ef@gmail.com
[2] https://github.com/KDE/heaptrack/blob/master/src/track/libheaptrack.h
[3] https://github.com/KDE/heaptrack/blob/master/src/track/heaptrack_api.h

--
regards, Andrei Lepikhov,
pgEdge