Re: Adding basic NUMA awareness - Preliminary feedback and outline for an extensible approach

On 7/8/25 03:55, Cédric Villemain wrote:
> Hi Andres,
>
>> Hi,
>>
>> On 2025-07-05 07:09:00 +0000, Cédric Villemain wrote:
>>> In my work on more careful PostgreSQL resource management, I've come
>>> to the
>>> conclusion that we should avoid pushing policy too deeply into the
>>> PostgreSQL core itself. Therefore, I'm quite skeptical about integrating
>>> NUMA-specific management directly into core PostgreSQL in such a way.
>>
>> I think it's actually the opposite - whenever we pushed stuff like this
>> outside of core it has hurt postgres substantially. Not having
>> replication in
>> core was a huge mistake. Not having HA management in core is probably the
>> biggest current adoption hurdle for postgres.
>>
>> To deal better with NUMA we need to improve memory placement and various
>> algorithms, in an interrelated way - that's pretty much impossible to do
>> outside of core.
>
> Except the backend pinning which is easy to achieve, thus my comment on
> the related patch.
> I'm not claiming NUMA memory and all should be managed outside of core
> (though I didn't read other patches yet).
>

But an "optimal backend placement" seems to very much depend on where we
placed the various pieces of shared memory. Which the external module
will have trouble following, I suspect.

I still don't have any idea what exactly would the external module do,
how would it decide where to place the backend. Can you describe some
use case with an example?

Assuming we want to actually pin tasks from within Postgres, what I
think might work is allowing modules to "advise" on where to place the
task. But the decision would still be done by core.

regards

--
Tomas Vondra