Re: Optimize planner memory consumption for huge arrays

From: Andrei Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org, Евгений Бредня <e(dot)brednya(at)postgrespro(dot)ru>
Subject: Re: Optimize planner memory consumption for huge arrays
Date: 2024-02-20 04:17:31
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 19/2/2024 20:47, Tomas Vondra wrote:
> On 9/8/23 07:11, Lepikhov Andrei wrote:
>> Just for comparison, without partitioning:
>> elems 1 1E1 1E2 1E3 1E4
>> master: 12kB 14kB 37kB 266kB 2.5MB
>> patched: 12kB 11.5kB 13kB 24kB 141kB
> These improvements look pretty nice, considering how simple the patch
> seems to be. I can't even imagine how much memory we'd need with even
> more partitions (say, 1000) if 100 partitions means 274MB.
> BTW when releasing memory in scalararraysel, wouldn't it be good to also
> free the elem_values/elem_nulls? I haven't tried and maybe it's not that
> significant amount.
Agree. Added into the next version of the patch.
Moreover, I see a slight planning speedup. Looking into the reason for
that, I discovered that it is because sometimes the planner utilizes the
same memory piece for the next array element. It finds this piece more
quickly than before that optimization.

Andrei Lepikhov
Postgres Professional

Attachment Content-Type Size
0001-Utilize-memory-in-scalararraysel-in-more-optimal-man.patch text/plain 2.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Shlok Kyal 2024-02-20 04:19:18 Re: speed up a logical replica setup
Previous Message jian he 2024-02-20 04:03:31 Re: POC, WIP: OR-clause support for indexes