Re: Load Distributed Checkpoints, final patch

Tom Lane wrote:
> Heikki Linnakangas <heikki(at)enterprisedb(dot)com> writes:
>> Here's latest revision of Itagaki-sans Load Distributed Checkpoints patch:
>
> Applied with some minor revisions to make some of the internal APIs a
> bit cleaner; mostly, it seemed like a good idea to replace all those
> bool parameters with a flag-bits approach, so that you could have
> something like "CHECKPOINT_FORCE | CHECKPOINT_WAIT" instead of
> "false, true, true, false" ...

Thanks.

> For the moment I removed all the debugging elog's in the patch.
> We still have Greg Smith's checkpoint logging patch to look at
> (which I suppose needs adjustment now), and that seems like the
> appropriate venue to consider what to put in.

Ok, I'll look at that next.

> One other closely connected item that might be worth looking at is the
> code for creating new future xlog segments (PreallocXlogFiles). Greg
> was griping upthread about xlog segment creation being a real
> performance drag. I realized that as we currently have it set up, the
> checkpoint code is next to useless for high-WAL-volume installations,
> because it only considers making *one* future XLOG segment. Once you've
> built up enough XLOG segments, the system isn't too bad about recycling
> them, but there will be a nasty startup transient where foreground
> processes have to stop and make the things. I wonder whether it would
> help if we (a) have the bgwriter call PreallocXlogFiles during its
> normal loop, and (b) back the slop in PreallocXlogFiles way off, so that
> it will make a future segment as soon as we start using the last
> existing segment, instead of only when we're nearly done. This would at
> least make it more likely that the bgwriter does the work instead of a
> foreground process. I'm hesitant to go much further than that, because
> I don't want to bloat the minimum disk footprint for low-volume
> installations, but the minimum footprint is really 2 xlog files anyway...

That seems like a good idea. It might also become a problem if you have
WAL archiving set up and the archiving falls behind so that existing log
files are not recycled fast enough.

The comment in PreallocXlogFiles is out of date:

> /*
> * Preallocate log files beyond the specified log endpoint, according to
> * the XLOGfile user parameter.
> */

As you pointed out, it only preallocates one log file. And there is no
XLOGfile mentioned anywhere else in the source tree.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com