Skip site navigation (1) Skip section navigation (2)

Re: User concurrency thresholding: where do I look?

From: "Jignesh K(dot) Shah" <J(dot)K(dot)Shah(at)Sun(dot)COM>
To: "Jignesh K(dot) Shah" <J(dot)K(dot)Shah(at)Sun(dot)COM>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-performance(at)postgresql(dot)org, Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Subject: Re: User concurrency thresholding: where do I look?
Date: 2007-07-20 18:18:05
Message-ID: 46A0FC5D.40001@sun.com (view raw or flat)
Thread:
Lists: pgsql-performance
I forgot to add one more piece of information.. I also tried the same 
test with 64-bit postgresql with 6GB shared_buffers and results are the 
same it drops around the same point which to me sounds like a bottleneck..

More later

-Jignesh


Jignesh K. Shah wrote:
> Awww Josh,
>
> I was just enjoying the chat on the picket fence! :-)
>
> Anyway the workload is mixed (reads,writes) with simple to medium 
> queries. The workload is known to scale well. But inorder to provide 
> substantial input I am still trying to eliminate things that can 
> bottleneck.  Currently I have eliminated CPU (plenty free) , RAM 
> (memory  is 48GB RAM in this server for  a 32-bit postgresql 
> instance), IO Storage (used the free ram to do /tmp database to 
> eliminate IO)  and am still trying to eliminate any network 
> bottlenecks to say for sure we have a problem in PostgreSQL. But yes 
> till that final thing is confirmed (network which can very well be the 
> case) it could be a problem somewhere else. However the thing that 
> worries me is more of the big drop instead of remaining constant out 
> there..
>
> Anyway more on this within a day or two once I add more network nics 
> between the systems to eliminate network problems (even though stats 
> dont show them as problems right now) and also reduce malloc lock 
> penalties if any.
>
> As for other questions:
>
> max_locks_per_transactions is set to default (10 I believe) increasing 
> it still seems to degrade overall throughput number.
>
> max_connections is set to 1500 for now till I get decent scaling till 
> 1400-1500 users.
>
> There are no hard failures reported anywhere. Log min durations does 
> show that queries are now slowing down and taking longer.
>
> OS is not swapping and also eliminated IO  by putting the whole 
> database on /tmp
>
> So while I finish adding more network connections between the two 
> systems (need to get cards) do enjoy the following URL :-)
>
> http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070703-00073.html 
>
>
> Of course all postgresql.conf still remains from the old test so no 
> flames on that one again :-)
>
> Regards,
> Jignesh
>
>
>
>
> Josh Berkus wrote:
>> Tom,
>>
>>  
>>> Well, if the load is a lot of short writing transactions then you'd
>>> expect the throughput to depend on how fast stuff can be pushed down to
>>> WAL.  What have you got wal_buffers set to?  Are you using a commit
>>> delay?  What's the I/O system anyway (any BB write cache on the WAL
>>> disk?) and what wal sync method are you using?
>>>     
>>
>> You know, I think Jignesh needs to me on this list so I can stop 
>> relaying questions on a workload I didn't design.  Let me get him.
>>
>>   
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>               http://archives.postgresql.org

In response to

pgsql-performance by date

Next:From: Tom LaneDate: 2007-07-20 18:23:51
Subject: Re: User concurrency thresholding: where do I look?
Previous:From: Tom LaneDate: 2007-07-20 18:12:26
Subject: Re: Simple query showing 270 hours of CPU time

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group