Skip site navigation (1) Skip section navigation (2)

Load Distributed Checkpoints, final patch

From: Heikki Linnakangas <heikki(at)enterprisedb(dot)com>
To: Patches <pgsql-patches(at)postgresql(dot)org>
Subject: Load Distributed Checkpoints, final patch
Date: 2007-06-26 18:49:15
Message-ID: 46815FAB.7030205@enterprisedb.com (view raw or flat)
Thread:
Lists: pgsql-patches
Here's latest revision of Itagaki-sans Load Distributed Checkpoints patch:

* bgwriter all-scan is gone. We might or might not improve the LRU-sweep 
later so that it can perform any duties the all-sweep might have had 
besides reducing the impact of a checkpoint.

* one new GUC variable, called checkpoint_completion_target. Default is 
0.5, which should be more than enough to smooth checkpoints on a system 
that's not already overloaded. It's also not too large to hurt recovery 
times too much on a system that's not already struggling to hit its 
recovery time requirements. You can set it to 0 if you want the old 
checkpoint behavior for some reason. Maximum is 0.9, to leave some 
headroom for fsync and any other things that need to happen during a 
checkpoint.

* The minimum rate we write at during a checkpoint is 1 page / 
bgwriter_delay.

* Added a paragraph to user manual to describe the feature. Also updated 
the formula for expected number of WAL segments, new formula is (2 + 
checkpoint_completion_target) * checkpoint_segments + 1. I believe the 
comments in xlog.c regarding XLOGfileslop are still valid.

* The signaling in bgwriter.c is based on a spinlock. Tom advised to not 
use the spinlock when not strictly necessary, but IMHO it's easier to 
understand this way. Feel free to revert that when committing if you 
disagree.

* The algorithm for estimating progress wrt. checkpoint_segments is the 
same as before. Bursty WAL activity will lead to bursty checkpoint 
activity, but I wanted to keep it simple for now. In any case, the I/O 
rate will be smoother than without the patch.

* There's some DEBUG elogs which we might want to replace with better 
ones later, per the patch in the patch queue by Greg Smith. The ones 
that are there now are useful for testing this feature, but are a bit 
crude for DBAs to use.

Barring any objections from committer, I'm finished with this patch.

I'm scheduling more DBT-2 tests at a high # of warehouses per Greg 
Smith's suggestion just to see what happens, but I doubt that will 
change my mind on the above decisions.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

Attachment: ldc-justwrites-6.patch
Description: text/x-diff (56.4 KB)

Responses

pgsql-patches by date

Next:From: Michael GlaesemannDate: 2007-06-26 19:08:46
Subject: Re: Load Distributed Checkpoints, final patch
Previous:From: Greg SmithDate: 2007-06-26 18:22:16
Subject: Re: Load Distributed Checkpoints, take 3

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group