Re: Load Distributed Checkpoints, take 3

Heikki Linnakangas <heikki(at)enterprisedb(dot)com> writes:
> Tom Lane wrote:
>> ... that's what the LRU scan is for.

> Yeah, except the LRU scan is not doing a very good job at that. It will
> ignore buffers with usage_count > 0, and it only scans
> bgwriter_lru_percent buffers ahead of the clock hand.

Which is exactly in sync with which buffers are actually candidates for
replacement. It could be looking further ahead of the clock hand, as
per my previous suggestion, but there is no need to push out buffers
with usage_count > 0 until after the sweep has decremented that to 0.
Doing so would tend to create excess I/O --- it makes it more likely
that multiple page-dirtying events on a page will result in separate
writes instead of a single write.

> One pathological case is a COPY of a table slightly smaller than
> shared_buffers. That will fill the buffer cache. If you then have a
> checkpoint, and after that a SELECT COUNT(*), or a VACUUM, the buffer
> cache will be full of pages with just hint-bit-updates, but no WAL
> activity since last checkpoint.

This argument supposes that the bgwriter will do nothing while the COPY
is proceeding.

regards, tom lane