Re: Optimize planner memory consumption for huge arrays

On 20/2/2024 04:51, Tom Lane wrote:
> Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> writes:
>> On 2/19/24 16:45, Tom Lane wrote:
>>> Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> writes:
>>>> For example, I don't think we expect selectivity functions to allocate
>>>> long-lived objects, right? So maybe we could run them in a dedicated
>>>> memory context, and reset it aggressively (after each call).
> Here's a quick and probably-incomplete implementation of that idea.
> I've not tried to study its effects on memory consumption, just made
> sure it passes check-world.
Thanks for the sketch. The trick with the planner_tmp_cxt_depth
especially looks interesting.
I think we should design small memory contexts - one per scalable
direction of memory utilization, like selectivity or partitioning
(appending ?).
My coding experience shows that short-lived GEQO memory context forces
people to learn on Postgres internals more precisely :).

--
regards,
Andrei Lepikhov
Postgres Professional