Re: getting the most of out multi-core systems for repeated complex SELECT statements

On Thu, Feb 3, 2011 at 9:19 PM, Andy Colson <andy(at)squeakycode(dot)net> wrote:
> On 02/03/2011 10:00 PM, Greg Smith wrote:
>>
>> Andy Colson wrote:
>>>
>>> Cpu's wont get faster, but HD's and SSD's will. To have one database
>>> connection, which runs one query, run fast, it's going to need multi-core
>>> support.
>>
>> My point was that situations where people need to run one query on one
>> database connection that aren't in fact limited by disk I/O are far less
>> common than people think. My troublesome database servers aren't ones with a
>> single CPU at its max but wishing there were more workers, they're the ones
>> that have >25% waiting for I/O. And even that crowd is still a subset,
>> distinct from people who don't care about the speed of any one core, they
>> need lots of connections to go at once.
>>
>
> Yes, I agree... for today.  If you gaze into 5 years... double the core
> count (but not the speed), double the IO rate.  What do you see?

I run a cluster of pg servers under slony replication, and we have 112
cores between three servers, soon to go to 144 cores. We have no need
for individual queries to span the cores, honestly. Our real limit is
the ability get all those cores working at the same time on individual
queries efficiently without thundering herd issues. Yeah, it's only
one datapoint, but for us, with a lot of cores, we need each one to
run one query as fast as it can.