Re: Background LRU Writer/free list

On Wed, Apr 18, 2007 at 09:09:11AM -0400, Greg Smith wrote:
> I'm mostly done with my review of the "Automatic adjustment of
> bgwriter_lru_maxpages" patch. In addition to issues already brought up
> with that code, there are some small things that need to be done to merge
> it with the recent pg_stat_bgwriter patch, and I have some concerns about
> its unbounded scanning of the buffer pool; I'll write that up in more
> detail or just submit an improved patch as I get time this week.
>
> But there's a fundamental question that has been bugging me, and I think
> it impacts the direction that code should take. Unless I'm missing
> something in my reading, buffers written out by the LRU writer aren't ever
> put onto the free list. I assume this is to stop from prematurely
> removing buffers that contain useful data. In cases where a substantial
> percentage of the buffer cache is dirty, the LRU writer has to scan a
> significant portion of the pool looking for one of the rare clean buffers,
> then write it out. When a client goes to grab a free buffer afterward, it
> has to scan the same section of the pool to find the now clean buffer,
> which seems redundant.
>
> With the new patch, the LRU writer is fairly well bounded in that it
> doesn't write out more than it thinks it will need; you shouldn't get into
> a situation where many more pages are written than will be used in the
> near future. Given that mindset, shouldn't pages the LRU scan writes just
> get moved onto the free list?

I've wondered the same thing myself.

If we're worried about freeing pages that we might want back, we could
change the code so that ReadBuffer would also look at the free list if
it couldn't find a page before going to the OS for it.

So if you make this change will BgBufferSync start incrementing
StrategyControl->nextVictimBuffer and decrementing buf->usage_count like
StrategyGetBuffer does now?
--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)