Re: Foreground vacuum and buffer access strategy

On Wed, Aug 14, 2013 at 1:41 AM, Greg Stark <stark(at)mit(dot)edu> wrote:
> On Tue, Aug 13, 2013 at 3:45 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>
>> I'm not sure what the right thing to do here is, but I definitely
>> agree there's a problem. There are definitely cases where people want
>> or indeed need to vacuum as fast as possible, and using a small ring
>> buffer is not the way to do that.
>
> I'm not convinced using a ring buffer is necessarily that bad even if
> you want to vacuum as fast as possible. The reason we use a small ring
> buffer is to avoid poisoning the entire cache with vacuum pages, not
> to throttle the speed of vacuum by introducing synchronous wal
> flushes.
>
> I think we should increase the size of the ring buffer if we hit a
> synchronous wal buffer flush and there is less than some amount of wal
> pending.
It will be better if the decision to increase ring buffer also
consider other activity, otherwise
it can lead to more I/O due to buffer replacements by backend.
I am not sure currently there is any way to check that, but if we
maintain buffers on free list, then
it can be used to check the current activity (if there are enough
buffers on free list, then ring size can be increased as it
is an indication that the system is relatively less busy).

> That amount is the relevant thing people might want to limit
> to avoid slowing down other transaction commits. The walwriter might
> even provide a relevant knob already for how much wal should be the
> maximum pending.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com