Skip site navigation (1) Skip section navigation (2)

Re: Bgwriter LRU cleaning: we've been going at this all wrong

From: Greg Smith <gsmith(at)gregsmith(dot)com>
To: pgsql-hackers(at)postgreSQL(dot)org
Subject: Re: Bgwriter LRU cleaning: we've been going at this all wrong
Date: 2007-06-27 04:48:25
Message-ID: Pine.GSO.4.64.0706270021100.10954@westnet.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Tue, 26 Jun 2007, Heikki Linnakangas wrote:

> How much of the buffer cache do you think we should try to keep 
> clean? And how large a percentage of the buffer cache do you think have 
> usage_count=0 at any given point in time?

What I discovered is that most of the really bad checkpoint pause cases I 
ran into involved most of the buffer cache being dirty while also having a 
non-zero usage count, which left the background writer hard-pressed to 
work usefully (the LRU writer couldn't do anything, and the all-scan was 
writing wastefully).  I was seeing >90% dirty+usage_count>0 in the really 
ugly spots.

What I like about Tom's idea is that it will keep the LRU writer in the 
best possible zone for that case (writing out madly right behind the LRU 
sweeper as counts get to zero) while still being fine on the more normal 
ones like you describe.  In particular, it should cut down on how much 
client backends write buffers in an overloaded case considerably.

> That will vary widely depending on your workload, of course, but keeping 1/4 
> of the buffer cache clean seems like overkill to me.

What may need to happen here is to add Tom's approach, but perhaps 
restrain it using the current auto-tuning LRU patch's method of estimating 
how many clean buffers are needed in the near future.  Particularly on 
large buffer caches, the idea of getting so far ahead of the sweep that 
you're looping all the away around and following right behind the clock 
sweep point may be overkill, but I think it will help enormously on 
smaller caches that are often very dirty.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

In response to

Responses

pgsql-hackers by date

Next:From: Greg SmithDate: 2007-06-27 04:57:30
Subject: Re: Bgwriter LRU cleaning: we've been going at this all wrong
Previous:From: ITAGAKI TakahiroDate: 2007-06-27 02:31:50
Subject: Re: Bgwriter LRU cleaning: we've been going at this all wrong

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group