Re: Initial 9.2 pgbench write results

On Thu, Feb 23, 2012 at 8:44 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> On 02/23/2012 07:36 AM, Simon Riggs wrote:
>>
>> Since scan_whole_pool_milliseconds is set to 2 minutes we scan the
>> whole bufferpool every 2 minutes, no matter how big the bufferpool,
>> even when nothing else is happening. Not cool.
>
>
> It's not quite that bad.  Once the BGW has circled around the whole buffer
> pool, such that it's swept so far ahead it's reached the clock sweep
> strategy point, it stops.  So when the system is idle, it creeps forward
> until it's scanned the pool once.  Then, it still wakes up regularly, but
> the computation of the bufs_to_lap lap number will reach 0.  That aborts
> running the main buffer scanning loop, so it only burns a bit of CPU time
> and a lock on BufFreelistLock each time it wakes--both of which are surely
> to spare if the system is idle.  I can agree with your power management
> argument, I don't see much of a performance win from eliminating this bit.

The behaviour is wrong though, because we're scanning for too long
when the system goes quiet and then we wake up again too quickly - as
soon as a new buffer allocation happens.

We don't need to clean the complete bufferpool in 2 minutes. That's
exactly the thing checkpoint does and we slowed that down so it didn't
do that. So we're still writing way too much.

So the proposal was to make it scan only 10% of the bufferpool, not
100%, then sleep. We only need some clean buffers, we don't need *all*
buffers clean, especially on very large shared_buffers. And we should
wake up only when we see an effect on user backends, i.e. a dirty
write - which is the event the bgwriter is designed to avoid.

The last bit is the key - waking up only when a dirty write occurs. If
they aren't happening we literally don't need the bgwriter - as your
tests show.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services