Re: Adding more memory = hugh cpu load

From: Leonardo Francalanci <m_lists(at)yahoo(dot)it>
To: Greg Smith <greg(at)2ndQuadrant(dot)com>, "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Adding more memory = hugh cpu load
Date: 2011-10-11 08:57:57
Message-ID: 1318323477.19815.YahooMailNeo@web29005.mail.ird.yahoo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

> checkpoint_completion_targets spreads out the writes to disk.  PostgreSQL 

> doesn't make any attempt yet to spread out the sync calls.  On a busy
> server, what can happen is that the whole OS write cache fills with dirty
> data--none of which is written out to disk because of the high kernel
> threshold--and then it all slams onto disk fast once the checkpoint starts
> executing sync calls.  Lowering the size of the Linux write cache helps with
> that a lot, but can't quite eliminate the problem.

Ok, I understand now

> Except that PostgreSQL doesn't look after the fsync calls yet.  I wrote a 

> patch for 9.1 that spread out the sync calls, similarly to how the writes are
> spread out now.  I wasn't able to prove an improvement sufficient to commit
> the result.  In the Linux case, the OS has more information to work with about
> how to schedule I/O efficiently given how the hardware is acting, and it's
> not possible for PostgreSQL to know all that--not without duplicating a large
> portion of the kernel development work at least.  Right now, relying the kernel
> means that any improvements there magically apply to any PostgreSQL version.  So
> far the results there have been beating out improvements made to the database
> fast enough that it's hard to innovate in this area within Postgres.

Ok; thank you very much for the explanation.

In fact, shouldn't those things be explained in the "WAL Configuration" section
of the manual? It looks as important as configuring Postgresql itself...
And: that applies to Linux. What about other OS, such as Solaris and FreeBSD?

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message James Cranch 2011-10-11 10:16:06 Rapidly finding maximal rows
Previous Message Greg Smith 2011-10-11 06:50:46 Re: Adding more memory = hugh cpu load