Skip site navigation (1) Skip section navigation (2)

Re: User concurrency thresholding: where do I look?

From: "Jignesh K(dot) Shah" <J(dot)K(dot)Shah(at)Sun(dot)COM>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-performance(at)postgresql(dot)org, Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Subject: Re: User concurrency thresholding: where do I look?
Date: 2007-07-20 18:08:49
Message-ID: 46A0FA31.2080208@sun.com (view raw or flat)
Thread:
Lists: pgsql-performance
Awww Josh,

I was just enjoying the chat on the picket fence! :-)

Anyway the workload is mixed (reads,writes) with simple to medium 
queries. The workload is known to scale well. But inorder to provide 
substantial input I am still trying to eliminate things that can 
bottleneck.  Currently I have eliminated CPU (plenty free) , RAM 
(memory  is 48GB RAM in this server for  a 32-bit postgresql instance), 
IO Storage (used the free ram to do /tmp database to eliminate IO)  and 
am still trying to eliminate any network bottlenecks to say for sure we 
have a problem in PostgreSQL. But yes till that final thing is confirmed 
(network which can very well be the case) it could be a problem 
somewhere else. However the thing that worries me is more of the big 
drop instead of remaining constant out there..

Anyway more on this within a day or two once I add more network nics 
between the systems to eliminate network problems (even though stats 
dont show them as problems right now) and also reduce malloc lock 
penalties if any.

As for other questions:

max_locks_per_transactions is set to default (10 I believe) increasing 
it still seems to degrade overall throughput number.

max_connections is set to 1500 for now till I get decent scaling till 
1400-1500 users.

There are no hard failures reported anywhere. Log min durations does 
show that queries are now slowing down and taking longer.

OS is not swapping and also eliminated IO  by putting the whole database 
on /tmp

So while I finish adding more network connections between the two 
systems (need to get cards) do enjoy the following URL :-)

http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070703-00073.html

Of course all postgresql.conf still remains from the old test so no 
flames on that one again :-)

Regards,
Jignesh




Josh Berkus wrote:
> Tom,
>
>   
>> Well, if the load is a lot of short writing transactions then you'd
>> expect the throughput to depend on how fast stuff can be pushed down to
>> WAL.  What have you got wal_buffers set to?  Are you using a commit
>> delay?  What's the I/O system anyway (any BB write cache on the WAL
>> disk?) and what wal sync method are you using?
>>     
>
> You know, I think Jignesh needs to me on this list so I can stop relaying 
> questions on a workload I didn't design.  Let me get him.
>
>   

In response to

Responses

pgsql-performance by date

Next:From: Tom LaneDate: 2007-07-20 18:12:26
Subject: Re: Simple query showing 270 hours of CPU time
Previous:From: Dan HarrisDate: 2007-07-20 17:18:35
Subject: Re: Simple query showing 270 hours of CPU time

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group