Re: Bgwriter LRU cleaning: we've been going at this all wrong

On Tue, 26 Jun 2007, Tom Lane wrote:

> It should try to be behind the sweep, ie, so far ahead that it's lapped
> the clock sweep and is trailing along right behind it, cleaning buffers
> immediately after their usage_count falls to zero. All the rest of the
> buffer arena is either clean or has positive usage_count.

I've said before here that something has to fundamentally change with the
LRU writer for it to ever be really useful, because most of the time it's
executing over pages with a positive usage_count as you say here. One
idea I threw out before was to have it premptively lower the usage counts
as it scans ahead of the sweep point and then add the pages to the free
list, which you rightly had some issues with. This suggestion of a change
so you'd expect it to follow right behind the sweep point sounds like a
better plan that should result in even less client back-end writes, and I
really like a plan that finally casts the LRU writer control parameter in
a MB/s context.

(Some pointers to your comments when we've gone over this neighborhood
before: http://archives.postgresql.org/pgsql-hackers/2007-03/msg00642.php
http://archives.postgresql.org/pgsql-hackers/2007-04/msg00799.php )

I broke Itagaki-san's patch into two pieces when I was doing the review
cleanup on it specifically to make it easier to tinker with this part
without losing some of its other neat features. Heikki, did you do
anything with that LRU adjustment patch since I sent it out:
http://archives.postgresql.org/pgsql-patches/2007-05/msg00142.php

I already fixed the race condition bug you found in my version of the
code.

Unless someone else has a burning desire to implement Tom's idea faster
than me, I should be to build this new implementation myself in the next
couple of days. I still have the test environment leftover from the last
time I worked on this code, and I think everybody else who could handle
this job has more important higher-level things they could be working on
instead.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD