Re: 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

From: Sergey Koposov <koposov(at)ast(dot)cam(dot)ac(dot)uk>
To: Florian Pflug <fgp(at)phlo(dot)org>
Cc: Merlin Moncure <mmoncure(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>
Subject: Re: 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile
Date: 2012-05-31 00:26:27
Message-ID: alpine.LRH.2.02.1205310103190.6351@calx046.ast.cam.ac.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, 31 May 2012, Florian Pflug wrote:

> Wait, so performance *increased* by spreading the backends out over as
> many dies as possible, not by using as few as possible? That'd be
> exactly the opposite of what I'd have expected. (I'm assuming that cores
> on one die have ascending ids on linux. If you could post the contents
> of /proc/cpuinfo, we could verify that)

Yes, you are correct. And I also can confirm that the cpus in the cpuinfo
are ordered by "physical id" e.g. they go like
0 0 0 0 0 0 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3

I did a specific test with just 6 threads (== number of cores per cpu)
and ran it on a single phys cpu, it took ~ 12 seconds for each thread,
and when I tried to spread it across 4 cpus it took 7-9 seconds per
thread. But all these numbers are anyway significantly better then when I
didn't use taskset. Which probably means without it the processes were
jumping from core to core ? ...

>> Because still the slowdown was caused by locking. If there wouldn't be
>> locking there wouldn't be any problems (as demonstrated a while ago by
>> just cat'ting the files in multiple threads).
>
> Yup, we'll have to figure out a way to reduce the locking overhead. 9.2
> already scales much better to a large number of cores than previous
> versions did, but your test case shows that there's still room for
> improvement.

Yes, and unfortunately these scaling problems in read-only cpu
bound queries, where naively I wound't expect any.

Thanks,

Sergey

*****************************************************
Sergey E. Koposov, PhD, Research Associate
Institute of Astronomy, University of Cambridge
Madingley road, CB3 0HA, Cambridge, UK
Tel: +44-1223-337-551 Web: http://www.ast.cam.ac.uk/~koposov/

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Janes 2012-05-31 00:46:31 Re: 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile
Previous Message Tatsuo Ishii 2012-05-31 00:20:43 Re: pg_dump and thousands of schemas