Re: Optimize planner memory consumption for huge arrays

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: Lepikhov Andrei <a(dot)lepikhov(at)postgrespro(dot)ru>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Евгений Бредня <e(dot)brednya(at)postgrespro(dot)ru>
Subject: Re: Optimize planner memory consumption for huge arrays
Date: 2024-02-26 15:20:58
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> writes:
> On 2/25/24 17:29, Tom Lane wrote:
>> Yeah. Also: once we had such an idea, it'd be very tempting to apply
>> it to other frequently-reset contexts like the executor's per-tuple
>> evaluation contexts. I'm not quite prepared to argue that
>> MemoryContextReset should just act that way all the time ... but
>> it's sure interesting to think about.

> Do the context resets consume enough time to make this measurable?

I think they do. We previously invented the "isReset" mechanism to
eliminate work in the case of exactly zero allocations since the
last reset, and that made a very measurable difference at the time,
even though you'd think the amount of work saved would be negligible.
This idea seems like it might be able to supersede that one and win
in a larger fraction of cases.

> +1 to disable this optimization in assert-enabled builds. I guess we'd
> invent a new constant to disable it, and tie it to USE_ASSERT_CHECKING
> (similar to CLOBBER_FREED_MEMORY, for example).

> Thinking about CLOBBER_FREED_MEMORY, could it be useful to still clobber
> the memory, even if we don't actually reset the context?

I think in any case where we were trying to support debugging, we'd
just disable the optimization, so that reset always resets.

regards, tom lane

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andy Fan 2024-02-26 15:29:33 Re: Shared detoast Datum proposal
Previous Message Alvaro Herrera 2024-02-26 15:13:11 Re: Running the fdw test from the terminal crashes into the core-dump