Re: Bgwriter LRU cleaning: we've been going at this all wrong

From: ITAGAKI Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
To: Heikki Linnakangas <heikki(at)enterprisedb(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgreSQL(dot)org
Subject: Re: Bgwriter LRU cleaning: we've been going at this all wrong
Date: 2007-06-27 02:31:50
Message-ID: 20070627110859.6410.ITAGAKI.TAKAHIRO@oss.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Heikki Linnakangas <heikki(at)enterprisedb(dot)com> wrote:

> Tom Lane wrote:
> > In fact, the notion of the bgwriter's cleaning scan being "in front of"
> > the clock sweep is entirely backward. It should try to be behind the
> > sweep, ie, so far ahead that it's lapped the clock sweep and is trailing
> > along right behind it, cleaning buffers immediately after their
> > usage_count falls to zero. All the rest of the buffer arena is either
> > clean or has positive usage_count.
>
> That will vary widely depending on your workload, of course, but keeping
> 1/4 of the buffer cache clean seems like overkill to me. If any of those
> buffers are re-dirtied after we write them, the write was a waste of time.

Agreed intuitively, but I don't know how offen backends change usage_count
0 to 1. If the rate is high, backward-bgwriter would not work. It seems to
happen frequently when we use large shared buffers.

I read Tom is changing the bgwriter LRU policy from "clean dirty pages
recycled soon" to "clean dirty pages just when they turn out to be less
frequently used", right? I have another thought -- advancing bgwriter's
sweep-startpoint a little ahead.

[buf] 0 lru X bgw-start N
|-----|----------->|-----------------------------|

I think X=0 is in the current behavior and X=N is in the backward-bgwriter.
Are there any other appropriate values for X? It might be good to use
statistics information about buffer usage to modify X runtime.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Smith 2007-06-27 04:48:25 Re: Bgwriter LRU cleaning: we've been going at this all wrong
Previous Message Jeremy Drake 2007-06-27 02:21:43 Re: Bugtraq: Having Fun With PostgreSQL