Re: Memory Context Info dump

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Memory Context Info dump
Date: 2015-09-08 15:30:16
Message-ID: 16188.1441726216@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com> writes:
> In our customer environment as well as during development, we have observed that many time we need to get details of memory used by each contexts in order to analyze the memory consumption/leak.
> So I would like to propose one contrib function interface, which will dump the whole memory context into a file.

Under what circumstances would you invoke this? Not when you were already
out of memory.

> Use-case:
> In order to check the memory leak, this contrib function can be called before and after execution of query. Then comparison of these two dump will be helpful to further analyze.

That's pretty uncompelling, considering that intra-query memory leaks are
one of the major concerns. The evidence would be gone before you could
capture it.

> 1. Create one contrib function dump_memctxt_info, which is when executed, it will call the existing function MemoryContextStats.
> 2. The file to dump will be created in the following format:
> context_<process id>_<timestamp>.dump

Dumping to a file seems like probably the second least friendly API you
could devise ... although I'm not sure how to do better offhand. The
existing code is made to not need to allocate any more memory, and
anything that could return data to SQL would have to do that.

I don't disagree that it would be useful to have better info available in
this area, but this patch seems like a quick hack rather than a useful
tool.

As an example of potentially-more-useful aids, I'm wondering about
tracking the high-water memory consumption of each memory context.
(This probably wouldn't be terribly expensive if it were done at the
granularity of malloc requests rather than individual pallocs.)
Then perhaps something to log a context's peak usage at context
destruction time, if it exceeds some threshold or other.

Our past discussions about tracking total usage in a context tree
would be relevant here too.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thom Brown 2015-09-08 15:47:36 Re: patch for geqo tweaks
Previous Message Thom Brown 2015-09-08 14:57:50 Re: jsonb_concat: make sure we always return a non-scalar value