Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] Sun Donated a Sun Fire T2000 to the PostgreSQL

From: Arjen van der Meijden <acmmailing(at)tweakers(dot)net>
To: josh(at)agliodbs(dot)com
Cc: pgsql-hackers(at)postgresql(dot)org, Robert Lor <Robert(dot)Lor(at)sun(dot)com>,pgsql-performance(at)postgresql(dot)org
Subject: Re: [HACKERS] Sun Donated a Sun Fire T2000 to the PostgreSQL
Date: 2006-06-18 09:17:54
Message-ID: 44951A42.6060007@tweakers.net (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
On 17-6-2006 1:24, Josh Berkus wrote:
> Arjen,
> 
>> I can already confirm very good scalability (with our workload) on
>> postgresql on that machine. We've been testing a 32thread/16G-version
>> and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores
>> (with all four threads enabled).
> 
> Keen.   We're trying to keep the linear scaling going up to 32 cores of 
> course (which doesn't happen, presently).  Would you be interested in 
> helping us troubleshoot some of the performance issues?

You can ask your questions, if I happen to do know the answer, you're a 
step further in the right direction.

But actually, I didn't do much to get this scalability... So I won't be 
of much help to you, its not that I spent hours on getting this performance.
I just started out with the "normal" attempts to get a good config. 
Currently the shared buffers is set to 30k. Larger settings didn't seem 
to differ much on our previous 4-core version, so I didn't even check it 
out on this one. I noticed I forgot to set the effective cache size to 
more than 6G for this one too, but since our database is smaller than 
that, that shouldn't make any difference. The work memory was increased 
a bit to 2K. So there are no magic tricks here.

I do have to add its a recent checkout of 8.2devel compiled using Sun 
Studio 11. It was compiled using this as CPPFLAGS: -xtarget=ultraT1 
-fast -xnolibmopt

The -xnolibmopt was added because we couldn't figure out why it yielded 
several linking errors at the end of the compilation when the -xlibmopt 
from -fast was enabled, so we disabled that particular setting from the 
-fast macro.


The workload generated is an abstraction and simplification of our 
website's workload, used for benchmarking. Its basically a news and 
price comparision site and it runs on LAMP (with the M of MySQL), i.e. a 
lot of light queries, many primary-key or indexed "foreign-key" lookups 
for little amounts of records. Some aggregations for summaries, etc. 
There are little writes and hardly any on the most read tables.
The database easily fits in memory, the total size of the actively read 
tables is about 3G.
This PostgreSQL-version is not a direct copy of the queries and tables, 
but I made an effort of getting it more PostgreSQL-minded as much as 
possible. I.e. I combined a few queries, I changed "boolean"-enum's in 
MySQL to real booleans in Postgres, I added specific indexes (including 
partials) etc.

We use apache+php as clients and just open X apache processes using 'ab' 
at the same time to generate various amounts of concurrent workloads. 
Solaris scales really well to higher concurrencies and PostgreSQL 
doesn't seem to have problems with it either in our workload.

So its not really a real-life scenario, but its not a synthetic 
benchmark either.

Here is a graph of our performance measured on PostgreSQL:
http://achelois.tweakers.net/~acm/pgsql-t2000/T2000-schaling-postgresql.png

What you see are three lines. Each represents the amount of total "page 
views" processed in 600 seconds for a specific amount of Niagara-cores 
(i.e. 1, 2, 4, 6 and 8). Each core had all its threads enabled, so its 
actually 4, 8, 16, 24 and 32 virtual cpu's you're looking at.
The "Max"-line displays the maximum generated "page views" on a specific 
core-amount for any concurrency, respectively: 5, 13, 35, 45 and 60.
The "Bij 50" is the amount of "page views" it generated with 50 
apache-processes working at the same time (on two dual xeon machines, so 
25 each). I took 50 a bit arbitrary but all core-configs seemed to do 
pretty well under that workload.

The "perfect" line is based on the "Max" value for 1 core and then just 
multiplied by the amount of cores to have a linear reference. The "Bij 
50" and the "perfect" line don't differ too much in color, but the 
top-one is the "perfect" line.

In the near future we'll be presenting an article on this on our 
website, although that will be in dutch the graphs should still be easy 
to read for you guys.
And because of that I can't promise too much detailed information until 
then.

I hope I clarified things a bit now, if not ask me about it,
Best regards,

Arjen

In response to

Responses

pgsql-performance by date

Next:From: Tim AllenDate: 2006-06-19 06:12:07
Subject: Re: SAN performance mystery
Previous:From: Andrew DunstanDate: 2006-06-17 21:46:40
Subject: Re: [HACKERS] Sun Donated a Sun Fire T2000 to the PostgreSQL

pgsql-hackers by date

Next:From: Douglas McNaughtDate: 2006-06-18 12:57:43
Subject: Re: Rethinking stats communication mechanisms
Previous:From: Greg StarkDate: 2006-06-18 07:42:24
Subject: Re: Rethinking stats communication mechanisms

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group