Re: Massive I/O spikes during checkpoint

On Tue, Jul 10, 2012 at 4:03 PM, David Kerr <dmk(at)mr-paradox(dot)net> wrote:

>
> On Jul 9, 2012, at 10:51 PM, Maxim Boguk wrote:
>
>
>>
>> But what appears to be happening is that all of the data is being written
>> out at the end of the checkpoint.
>>
>> This happens at every checkpoint while the system is under load.
>>
>> I get the feeling that this isn't the correct behavior and i've done
>> something wrong.
>>
>>
>>
> It's not an actual checkpoints.
> It's is a fsync after checkpoint which create write spikes hurting server.
>
> You should set sysctl vm.dirty_background_bytes and vm.dirty_bytes to
> reasonable low values
>
>
> So use bla_bytes instead of bla_ratio?
>
>
Yes because on 256GB server
echo 10 > /proc/sys/vm/dirty_ratio
is equivalent to 26Gb dirty_bytes

and
echo 5 >/proc/sys/vm/dirty_background_ratio
is equivalent to 13Gb dirty_background_bytes

It is really huge values.

So kernel doesn't start write any pages out in background before it has at
least 13Gb dirty pages in kernel memory.
And at end of the checkpoint kernel trying flush all dirty pages to disk.

Even echo 1 >/proc/sys/vm/dirty_background_ratio is too high value for
contemporary server.
That is why *_bytes controls added to kernel.

--
Maxim Boguk
Senior Postgresql DBA
http://www.postgresql-consulting.com/