Re: Optimize planner memory consumption for huge arrays

Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> writes:
> On 2/25/24 17:29, Tom Lane wrote:
>> Yeah. Also: once we had such an idea, it'd be very tempting to apply
>> it to other frequently-reset contexts like the executor's per-tuple
>> evaluation contexts. I'm not quite prepared to argue that
>> MemoryContextReset should just act that way all the time ... but
>> it's sure interesting to think about.

> Do the context resets consume enough time to make this measurable?

I think they do. We previously invented the "isReset" mechanism to
eliminate work in the case of exactly zero allocations since the
last reset, and that made a very measurable difference at the time,
even though you'd think the amount of work saved would be negligible.
This idea seems like it might be able to supersede that one and win
in a larger fraction of cases.

> +1 to disable this optimization in assert-enabled builds. I guess we'd
> invent a new constant to disable it, and tie it to USE_ASSERT_CHECKING
> (similar to CLOBBER_FREED_MEMORY, for example).

> Thinking about CLOBBER_FREED_MEMORY, could it be useful to still clobber
> the memory, even if we don't actually reset the context?

I think in any case where we were trying to support debugging, we'd
just disable the optimization, so that reset always resets.

regards, tom lane