Skip site navigation (1) Skip section navigation (2)

Re: high user cpu, massive SELECTs, no io waiting problem

From: Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>
To: "Strange, John W" <john(dot)w(dot)strange(at)jpmchase(dot)com>
Cc: Marti Raudsepp <marti(at)juffo(dot)org>, Thomas Pöhler <tp(at)turtle-entertainment(dot)de>, "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>, Felix Feinhals <ff(at)turtle-entertainment(dot)de>, Verteiler_A-Team <a-team(at)turtle-entertainment(dot)de>, Björn Metzdorf <bm(at)turtle-entertainment(dot)de>
Subject: Re: high user cpu, massive SELECTs, no io waiting problem
Date: 2011-02-16 16:02:43
Message-ID: AANLkTinbrka9o6mqckgy=sDLBAK71UEB1XcmYH0Zy7qu@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-performance
Yeah, at max load we are.  We're running quad 12 core AMD Magny Cours.
 Under max load all of our cores go about 20 to 30% red (i.e. kernel)
and we wind up waiting on the kernel much more.  Could be a mix of
context switching and waiting on memory, so it's just a guesstimate
I'm making based on previous testing with Greg Smith's memory
streaming test and familiarity with this system.

On Wed, Feb 16, 2011 at 8:53 AM, Strange, John W
<john(dot)w(dot)strange(at)jpmchase(dot)com> wrote:
> Scott, are you really moving that much data through memory, 70-80GB/sec is the limit of the new intel 7500 series in a best case scenario.
>
> - John
>
> -----Original Message-----
> From: pgsql-performance-owner(at)postgresql(dot)org [mailto:pgsql-performance-owner(at)postgresql(dot)org] On Behalf Of Scott Marlowe
> Sent: 16 February 2011 15:43
> To: Marti Raudsepp
> Cc: Thomas Pöhler; pgsql-performance(at)postgresql(dot)org; Felix Feinhals; Verteiler_A-Team; Björn Metzdorf
> Subject: Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem
>
> On Wed, Feb 16, 2011 at 6:44 AM, Marti Raudsepp <marti(at)juffo(dot)org> wrote:
>> On Tue, Feb 15, 2011 at 20:01, Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com> wrote:
>>> run htop and look for red.  if youi've got lots of red bar on each CPU
>>> but no io wait then it's waiting for memory access.
>>
>> I don't think this is true. AFAICT the red bar refers to "system
>> time", time that's spent in the kernel -- either in syscalls or kernel
>> background threads.
>
> My point being that if you've got a lot of RED it'll be the OS waiting
> for memory access.  Trust me, when we start to hit our memory
> bandwidth (in the 70 to 80 GB/s range) we start to get more and more
> red and more and more kernel wait time.
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
> This communication is for informational purposes only. It is not
> intended as an offer or solicitation for the purchase or sale of
> any financial instrument or as an official confirmation of any
> transaction. All market prices, data and other information are not
> warranted as to completeness or accuracy and are subject to change
> without notice. Any comments or statements made herein do not
> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
> and affiliates.
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law. If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED. Although this transmission and any
> attachments are believed to be free of any virus or other defect
> that might affect any computer system into which it is received and
> opened, it is the responsibility of the recipient to ensure that it
> is virus free and no responsibility is accepted by JPMorgan Chase &
> Co., its subsidiaries and affiliates, as applicable, for any loss
> or damage arising in any way from its use. If you received this
> transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard
> copy format. Thank you.
>
> Please refer to http://www.jpmorgan.com/pages/disclosures for
> disclosures relating to European legal entities.
>



-- 
To understand recursion, one must first understand recursion.

In response to

pgsql-performance by date

Next:From: Kevin GrittnerDate: 2011-02-16 16:08:54
Subject: Re: high user cpu, massive SELECTs, no io waiting problem
Previous:From: Strange, John WDate: 2011-02-16 15:53:47
Subject: Re: high user cpu, massive SELECTs, no io waiting problem

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group