Re: Auto-tune shared_buffers to use available huge pages

From: Anthonin Bonnefoy <anthonin(dot)bonnefoy(at)datadoghq(dot)com>
To: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Auto-tune shared_buffers to use available huge pages
Date: 2026-01-29 16:52:39
Message-ID: CAO6_Xqo0F8UhuShbET648o3yE5BhbZgvwCeUH9TA0N7i=onGVQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Jan 29, 2026 at 3:46 PM Ashutosh Bapat
<ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> You could write a script to find the optimal value of shared_buffers,
> that will consume memory optimally; auto-tuning it without starting
> the server.
>
> Isn't that sufficient?

That was my initial approach. I have such a script and built a mapping
between memory available and shared_buffers to request, one for each
PG version. But that breaks as soon as a user does something that
increases memory request, like increasing max_connections or adding an
extension.

Thus, I need to run this script before every PG starts to account for
those possible changes. With Kubernetes, this means:
- You use a PG Docker image, you need to clone it and override the
entry point to run the script before starting PG.
- The cluster may be managed by systems like patroni[1], so patroni
would need to be modified to do the auto-tuning.
- You may also use a Kubernetes operator to manage the cluster. AFAIK,
they don't provide a way to override how PG is started, the operators
would need to implement the auto-tuning logic.

If this auto-tune logic is directly implemented in PG, all kubernetes
users would be able to benefit from this, without having to hijack how
PG is started.

[1]: https://github.com/patroni/patroni

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2026-01-29 17:06:03 Re: refactor architecture-specific popcount code
Previous Message Florin Irion 2026-01-29 16:45:52 Re: [PATCH] pg_get_domain_ddl: DDL reconstruction function for CREATE DOMAIN statement