Re: Estimating HugePages Requirements?

From: "Bossart, Nathan" <bossartn(at)amazon(dot)com>
To: Justin Pryzby <pryzby(at)telsasoft(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Magnus Hagander <magnus(at)hagander(dot)net>, Mark Dilger <mark(dot)dilger(at)enterprisedb(dot)com>, Don Seiler <don(at)seiler(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Estimating HugePages Requirements?
Date: 2021-08-28 05:36:37
Message-ID: 82E1E183-691F-479E-B661-9BE8EC70BD21@amazon.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin pgsql-hackers

On 8/27/21, 7:01 PM, "Michael Paquier" <michael(at)paquier(dot)xyz> wrote:
> On Fri, Aug 27, 2021 at 08:16:40PM +0000, Bossart, Nathan wrote:
>> On 8/27/21, 12:39 PM, "Andres Freund" <andres(at)anarazel(dot)de> wrote:
>>> One thing I wonder is if this wouldn't better be dealt with in a more generic
>>> way. While this is the most problematic runtime computed GUC, it's not the
>>> only one. What if we introduced a new shared_memory_size GUC, and made
>>> --describe-config output it? Perhaps adding --describe-config=guc-name?
>>>
>>> I also wonder if we should output the number of hugepages needed instead of
>>> the "raw" bytes of shared memory. The whole business about figuring out the
>>> huge page size, dividing the shared memory size by that and then rounding up
>>> could be removed in that case. Due to huge_page_size it's not even immediately
>>> obvious which huge page size one should use...
>>
>> I like both of these ideas.
>
> That pretty much looks like -C in concept, isn't it? Except that you
> cannot get the actual total shared memory value because we'd do this
> operation before loading shared_preload_libraries and miss any amount
> asked by extensions. There is a problem similar when attempting to do
> postgres -C data_checksums, for example, which would output an
> incorrect value even if the cluster has data checksums enabled.

Attached is a hacky attempt at adding a shared_memory_size GUC in a
way that could be used with -C. This should include the amount of
shared memory requested by extensions, too. As long as huge_page_size
is nonzero, it seems easy enough to provide the number of huge pages
needed as well.

Nathan

Attachment Content-Type Size
0001-add-shared_memory_size-GUC.patch application/octet-stream 2.7 KB

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message andrew burns 2021-08-30 03:50:23
Previous Message Justin Pryzby 2021-08-28 03:57:22 Re: Estimating HugePages Requirements?

Browse pgsql-hackers by date

  From Date Subject
Next Message hubert depesz lubaczewski 2021-08-28 06:38:24 Re: Can we get rid of repeated queries from pg_dump?
Previous Message Justin Pryzby 2021-08-28 03:57:22 Re: Estimating HugePages Requirements?