Re: Random performance hit, unknown cause.

From: "Strange, John W" <john(dot)w(dot)strange(at)jpmorgan(dot)com>
To: "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Random performance hit, unknown cause.
Date: 2012-04-18 16:54:22
Message-ID: D785635498B68242A957B09272733DBF06DFF663@SCACMX007.exchad.jpmchase.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Check your pagecache settings, when doing heavy io writes of a large file you can basically force a linux box to completely stall. At some point once the pagecache has reached it's limit it'll force all IO to go sync basically from my understanding. We are still fighting with this but lots of changes in RH6 seem to address of lot of these issues.

grep -i dirty /proc/meminfo
cat /proc/sys/vm/
cat /proc/sys/vm/nr_pdflush_threads

Once the dirty pages reaches a really large size and the limit of pagecache your system should experience a pretty abrupt drop in performance. You should be able to avoid this by using sync writes, but we haven't had a chance to completely isolate and address this issue.

-----Original Message-----
From: pgsql-performance-owner(at)postgresql(dot)org [mailto:pgsql-performance-owner(at)postgresql(dot)org] On Behalf Of Claudio Freire
Sent: Thursday, April 12, 2012 1:50 PM
To: Brian Fehrle
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: [PERFORM] Random performance hit, unknown cause.

On Thu, Apr 12, 2012 at 3:41 PM, Brian Fehrle <brianf(at)consistentstate(dot)com> wrote:
> This morning, during our nightly backup process (where we grab a copy
> of the data directory), we started having this same issue. The main
> thing that I see in all of these is a high disk wait on the system.
> When we are performing 'well', the %wa from top is usually around 30%,
> and our load is around 12 - 15. This morning we saw a load  21 - 23,
> and an %wa jumping between 60% and 75%.
>
> The top process pretty much at all times is the WAL Sender Process, is
> this normal?

Sounds like vacuum to me.

--
Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message Andy Colson 2012-04-18 19:47:49 Re: scale up (postgresql vs mssql)
Previous Message Eyal Wilde 2012-04-18 07:32:07 Re: scale up (postgresql vs mssql)