Re: (full) Memory context dump considered harmful

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org, Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>
Subject: Re: (full) Memory context dump considered harmful
Date: 2015-08-21 22:00:41
Message-ID: 8021.1440194441@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I wrote:
> I thought a bit more about this. Generally, what you want to know about
> a given situation is which contexts have a whole lot of allocations
> and/or a whole lot of child contexts. What you suggest above won't work
> very well if the problem is buried more than about two levels down in
> the context tree. But suppose we add a parameter to memory context stats
> collection that is the maximum number of child contexts to print *per
> parent context*. If there are more than that, summarize the rest as per
> your suggestion.

Here's a draft patch along this line. After playing with it a bit,
I think that a wired-in limit of 100 children per level would be fine.
We could imagine adding a GUC for that, but I think it's overkill.

A couple of notes:

Since the summarization mechanism requires passing totals back up anyway,
I took the opportunity to add a "grand total" line to the end of the
printout. I'm half tempted to modify that to convert the numbers in the
grand total line to kilobytes or even MB, but it could be argued either way.

This assumes that the total memory allocation couldn't exceed the range of
size_t, a thing which is not promised by the C standard --- but AFAIK it
would be true on any modern machine (and really we were already making
such an assumption in AllocSetStats). Also, if we ever add a memory
context type that works fundamentally differently from AllocSet, we might
need to rethink what we compute/print as summary stats. I figure we can
cross that bridge when we come to it.

Comments?

regards, tom lane

Attachment Content-Type Size
summarize-memory-context-stats.patch text/x-diff 10.1 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Jim Nasby 2015-08-21 22:09:01 Re: pg_dump quietly ignore missing tables - is it bug?
Previous Message Jeff Janes 2015-08-21 21:21:04 Re: Patch for ginCombineData