Re: Straightforward changes for increased SMP scalability

Tom,

I'm happy to run some benchmarks to show the improvements with various
NUM_BUFFER_PARTITIONS settings. However, I want to make sure that this
is going to be useful. I can run 16 (base), 32, 64, 128 etc. type
increments, but I'm more concerned about the number of cores to use. Do
you have a suggestion for that? I can run with 1 to 32 cores. I had
planned to run a number of tests at 8 cores, but I can adjust to what
makes sense for the community.

David

-----Original Message-----
From: pgsql-hackers-owner(at)postgresql(dot)org
[mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Tom Lane
Sent: Monday, July 16, 2007 9:10 AM
To: Simon Riggs
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] Straightforward changes for increased SMP
scalability

"Simon Riggs" <simon(at)2ndquadrant(dot)com> writes:
> 2. Increase NUM_BUFFER_PARTITIONS from 16 to 256 (or higher).

Do you have any evidence to back up such a large increase?

This change is not free; at the very least it will break
contrib/pg_buffercache, which wants to lock all the partitions at once.
lwlock.c was designed on the assumption that only a pretty small number
of LWLocks would ever be held concurrently, and it will fall over.
I don't think fixing this would be as simple as increasing
MAX_SIMUL_LWLOCKS, because some of the algorithms are O(N^2).

I'd like to see numbers proving that there is useful incremental gain
from going above 32 or 64 partitions, before we start hacking to make
this work.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster