Re: Load Distributed Checkpoints, final patch

On Wed, 27 Jun 2007, Tom Lane wrote:

> Also, the question of redesigning the bgwriter's LRU scan is
> still open. I believe that's on Greg's plate, too.

Greg's plate was temporarily fried after his house was hit by lightening
yesterday. I just got everything back on-line again, so no coding
progress, but I think I finished assimilating your "epiphany" during that
time. Now I realize that what you're suggesting is that under healthy
low-load conditions, the LRU really should be able to keep up right behind
the clock sweep point. Noting how far behind it is serves as a
measurement of it failing to match the rate buffers that could be re-used
are being dirtied, and the only question is how fast and far it should try
to drive the point it has cleaned to forward when that happens.

> Once you've built up enough XLOG segments, the system isn't too bad
> about recycling them, but there will be a nasty startup transient where
> foreground processes have to stop and make the things.

Exactly. I found it problematic in four situations:

1) Slow checkpoint doesn't finish in time and new segments are being
created while the checkpoint is also busy (this is the really bad one)

2) Archive logger stop doing anything (usually because the archive disk is
filled) and nothing gets recycled until that's fixed.

2) checkpoint_segments is changed, so then performance is really sluggish
for a bit until all the segments are built back up again

3) You ran an early manual checkpoint which doesn't seem to recycle as
many segments usefully

Any change that would be more proactive about creating segments in these
situations than the current code would be a benefit, even though these are
not common paths people encounter.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD