Skip site navigation (1) Skip section navigation (2)

Re: Load Distributed Checkpoints, final patch

From: Heikki Linnakangas <heikki(at)enterprisedb(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Patches <pgsql-patches(at)postgresql(dot)org>, ITAGAKI Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>, Greg Smith <gsmith(at)gregsmith(dot)com>
Subject: Re: Load Distributed Checkpoints, final patch
Date: 2007-06-28 09:14:05
Message-ID: 46837BDD.2000705@enterprisedb.com (view raw or flat)
Thread:
Lists: pgsql-patches
Tom Lane wrote:
> Heikki Linnakangas <heikki(at)enterprisedb(dot)com> writes:
>> Here's latest revision of Itagaki-sans Load Distributed Checkpoints patch:
> 
> Applied with some minor revisions to make some of the internal APIs a
> bit cleaner; mostly, it seemed like a good idea to replace all those
> bool parameters with a flag-bits approach, so that you could have
> something like "CHECKPOINT_FORCE | CHECKPOINT_WAIT" instead of
> "false, true, true, false" ...

Thanks.

> For the moment I removed all the debugging elog's in the patch.
> We still have Greg Smith's checkpoint logging patch to look at
> (which I suppose needs adjustment now), and that seems like the
> appropriate venue to consider what to put in.

Ok, I'll look at that next.

> One other closely connected item that might be worth looking at is the
> code for creating new future xlog segments (PreallocXlogFiles).  Greg
> was griping upthread about xlog segment creation being a real
> performance drag.  I realized that as we currently have it set up, the
> checkpoint code is next to useless for high-WAL-volume installations,
> because it only considers making *one* future XLOG segment.  Once you've
> built up enough XLOG segments, the system isn't too bad about recycling
> them, but there will be a nasty startup transient where foreground
> processes have to stop and make the things.  I wonder whether it would
> help if we (a) have the bgwriter call PreallocXlogFiles during its
> normal loop, and (b) back the slop in PreallocXlogFiles way off, so that
> it will make a future segment as soon as we start using the last
> existing segment, instead of only when we're nearly done.  This would at
> least make it more likely that the bgwriter does the work instead of a
> foreground process.  I'm hesitant to go much further than that, because
> I don't want to bloat the minimum disk footprint for low-volume
> installations, but the minimum footprint is really 2 xlog files anyway...

That seems like a good idea. It might also become a problem if you have 
WAL archiving set up and the archiving falls behind so that existing log 
files are not recycled fast enough.

The comment in PreallocXlogFiles is out of date:

> /*
>  * Preallocate log files beyond the specified log endpoint, according to
>  * the XLOGfile user parameter.
>  */

As you pointed out, it only preallocates one log file. And there is no 
XLOGfile mentioned anywhere else in the source tree.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

In response to

Responses

pgsql-patches by date

Next:From: Jacob RiefDate: 2007-06-28 10:08:37
Subject: Re: SPI-header-files safe for C++-compiler
Previous:From: Neil ConwayDate: 2007-06-28 06:40:43
Subject: Re: psql: add volatility to \df+

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group