Re: RFC: Logging plan of the running query

From: torikoshia <torikoshia(at)oss(dot)nttdata(dot)com>
To: James Coleman <jtc331(at)gmail(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Greg Stark <stark(at)mit(dot)edu>, Ronan Dunklau <ronan(dot)dunklau(at)aiven(dot)io>, david(dot)christensen(at)crunchydata(dot)com, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Subject: Re: RFC: Logging plan of the running query
Date: 2023-08-17 14:02:22
Message-ID: af8b4c4752dcee3ca7284e7c15922d5d@oss.nttdata.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2023-06-16 01:34, James Coleman wrote:
> Attached is v28
> which sets ProcessLogQueryPlanInterruptActive to false in errfinish
> when necessary. Once built with those two patches I'm simply running
> `make check`.

With v28-0001 and v28-0002 patch, I confirmed backend processes consume
huge
amount of memory and under some environments they were terminated by OOM
killer.

This was because memory was allocated from existing memory contexts and
they
were not freed after ProcessLogQueryPlanInterrupt().
Updated the patch to use dedicated memory context for
ProcessLogQueryPlanInterrupt().

Applying attached patch and v28-0002 patch, `make check` successfully
completed after 20min and 50GB of logs on my environment.

>>> On 2023-06-15 01:48, James Coleman wrote:
>>> > The tests have been running since last night, but have been apparently
>>> > hung now for many hours.

I don't know if this has anything to do with the hung you faced, but I
thought
it might be possible that the large amount of memory usage resulted in
swapping, which caused a significant delay in processing.

If possible, I would be very grateful if you could try to reproduce this
with
the v29 patch.

--
Regards,

--
Atsushi Torikoshi
NTT DATA CORPORATION

Attachment Content-Type Size
v29-0001-Add-function-to-log-the-plan-of-the-query.patch text/x-diff 27.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peifeng Qiu 2023-08-17 14:11:47 Allow Index AM scan API to access information about LIMIT clause
Previous Message Frédéric Yhuel 2023-08-17 13:50:58 Re: Allow parallel plan for referential integrity checks?