Re: Load Distributed Checkpoints, take 3

Tom Lane wrote:
> I agree with removing the non-LRU
> part of the bgwriter's write logic though; that should simplify matters
> a bit and cut down the overall I/O volume.

On further thought, there is one workload where removing the non-LRU
part would be counterproductive:

If you have a system with a very bursty transaction rate, it's possible
that when it's time for a checkpoint, there hasn't been any WAL logged
activity since last checkpoint, so we skip it. When that happens, the
buffer cache might still be full of dirty pages, because of hint bit
updates. That still isn't a problem on it's own, but if you then do a
huge batch update, you have to flush those dirty pages at that point. It
would be better to trickle flush those dirty pages during the idle period.

So we might still want to keep the non-LRU scan. Or alternatively, when
the checkpoint is a no-op, we call BufferSync nevertheless. That's
effectively the same thing, except that BufferSync would be controlled
by the logic to estimate the deadline until next checkpoint, instead of
the current bgwriter_all_*-settings.

Greg, is this the kind of workload you're having, or is there some other
scenario you're worried about?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com