Re: Load Distributed Checkpoints test results

Heikki Linnakangas wrote:
> Here's results from a batch of test runs with LDC. This patch only
> spreads out the writes, fsyncs work as before. This patch also includes
> the optimization that we don't write buffers that were dirtied after
> starting the checkpoint.
>
> http://community.enterprisedb.com/ldc/
>
> See tests 276-280. 280 is the baseline with no patch attached, the
> others are with load distributed checkpoints with different values for
> checkpoint_write_percent. But after running the tests I noticed that the
> spreading was actually controlled by checkpoint_write_rate, which sets
> the minimum rate for the writes, so all those tests with the patch
> applied are effectively the same; the writes were spread over a period
> of 1 minute. I'll fix that setting and run more tests.

I ran another series of tests, with a less aggressive bgwriter_delay
setting, which also affects the minimum rate of the writes in the WIP
patch I used.

Now that the checkpoints are spread out more, the response times are
very smooth.

With the 40% checkpoint_write_percent setting, the checkpoints last ~3
minutes. About 85% of the buffer cache is dirty at the beginning of
checkpoints, and thanks to the optimization of not writing pages dirtied
after checkpoint start, only ~47% of those are actually written by the
checkpoint. That explains why the checkpoints only last ~3 minutes, and
not checkpoint_timeout*checkpoint_write_percent, which would be 6
minutes. The estimation of how much progress has been done and how much
is left doesn't take the gain from that optimization into account.

The sync phase only takes ~5 seconds. I'm very happy with these results.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com