Re: Allow pg_log_backend_memory_contexts() for postmaster

From: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Allow pg_log_backend_memory_contexts() for postmaster
Date: 2026-06-23 06:35:51
Message-ID: CAKZiRmxbmBcxt6LF--Y3tjwPuPippX5-xViSUWBKXAJoFm7aYA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Jun 22, 2026 at 9:36 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> Hi,
>
> On 2026-06-22 15:15:27 +0200, Jakub Wartak wrote:
> > During one of recent investigations about memory leaks in postmaster, I was
> > quite tired of issues related to gdb (it's seems very risky on postmaster,
> > requires for gdb to be installed and might cause friction).
> >
> > I was wondering why we couldn't suppport this out of the box. It looks like
> > useful addition to me, so here's the patch.
>
> I agree it'd be useful, but I think it'd be somewhat risky to allow.

The thing is that not having anything like that ready to use in first place
is even bigger risk in my book since last week. Repro of what really happened
(it was more like hourly cron in paralell to some connection activity, but
this below repro reliably crashes at least on RHEL 9.7/9.8):

1. people suspect postmaster leak
2. pgbench --connect -c 12 -j 12 edb -T 300 -P 1 # app, dozens of conns/sec
3. while : ; do gdb -p <postmasterPID> -batch -ex 'call
(void)MemoryContextStats((void *)TopMemoryContext)'
-ex detach; done

Interesingly it can cause downtime (after some time) when SIGUSR1 arrives:

Program received signal SIGUSR1, User defined signal 1.
MemoryContextStats (context=0x3b82490) at utils/mmgr/mcxt.c:834
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwind-on-signal on".
Evaluation of the expression containing the function
(MemoryContextStats) will be abandoned.
When the function is done executing, GDB will silently stop.
[Inferior 1 (process 18644) detached]
Unable to attach: program terminated with signal SIGSEGV, Segmentation fault.

with unwindonsignal it does not crash whole cluster, but that's the point
that people might be not aware (those were not my instructions, but I
was not aware of such postmaster/gdb/unwindonsignal behavior too, so I would
easily allow that through review). Even
https://wiki.postgresql.org/wiki/Developer_FAQ
-> `Examining backend memory use` does not seem to point out this stuff.

> If you have some sort of memory leak issue in postmaster,
> ProcessLogMemoryContextInterrupt() could take a while without responding
> to incoming client connections, responding to signals etc (e.g. because
> there are many memory contexts, or because the log destination is slow
> processing a lot of output). Maybe that would practically be ok, but I'm
> not sure it will.

Hm, before sending I've tried this:
a. fast loop of pg_log_backend_memory_contexts($postmasterPID)
b. pgbench --connect -c $high -j $high
and it does not seem to have negative effects. There are just few lines in
case of postmaster:

TopMemoryContext: 8192 total in 1 blocks; 6256 free (70 chunks); 1936 used
GUCMemoryContext: 24576 total in 2 blocks; 12496 free (11 chunks); 12080 used
GUC hash table: 32768 total in 3 blocks; 6384 free (6 chunks); 26384 used
Timezones: 112320 total in 2 blocks; 2640 free (0 chunks); 109680 used
Postmaster: 30176 total in 3 blocks; 12872 free (87 chunks); 17304 used
ident parser context: 1024 total in 1 blocks; 784 free (0 chunks); 240 used
hba parser context: 17408 total in 5 blocks; 7760 free (6 chunks); 9648 used
ErrorContext: 8192 total in 1 blocks; 7952 free (0 chunks); 240 used
Grand total: 234656 bytes in 18 blocks; 57144 free (180 chunks); 177512 used

But even still as you say, maybe not great idea, but we could have:
pg_log_backend_memory_contexts(pid)
pg_log_backend_memory_contexts(pid, i_am_sure::boolean)

and if you run pg_log_backend_memory_contexts(postmasterPID) it issues WARNING
to run pg_log_backend_memory_contexts(postmasterPI, true) if you are really
sure?

Also if I/O is slow for logfile is so slow, wouldn't other terrible issues?
(I saw once hanging syslog() but never write-buffered local log)

> At least on !EXEC_BACKEND systems you can get the data in a forked process,
> before the forked process does MemoryContextDelete(PostmasterContext). In
> theory you could rig things up so that the child logs the memory context
> information. However that currently is probably not doable because we won't
> accept interrupts before the MemoryContextDelete()...

Hm, interesting idea, so something like PG_PROCTYPE() for
B_POSTMASTER_MEMCTXDUMP? Would having feature like that not working on Win32,
be acceptable from project point of view?

I guess I need direction to point me to proper path here (there are 3-4 ideas
surfacing: accept risk, reject due to risks, extended with 2nd arg for
pg_log_backend_memory_contexts(), report through fork() and then exit).

-J.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2026-06-23 06:39:44 Re: (SQL/PGQ) Clean up orphaned properties when dropping a label
Previous Message wenhui qiu 2026-06-23 06:16:52 Re: Reduce LEFT/FULL JOIN to ANTI JOIN in more cases