Re: Redesigning checkpoint_segments

On 6/6/13 4:09 PM, Heikki Linnakangas wrote:
> On 06.06.2013 20:24, Josh Berkus wrote:
>>> Yeah, something like that :-). I was thinking of letting the estimate
>>> decrease like a moving average, but react to any increases immediately.
>>> Same thing we do in bgwriter to track buffer allocations:
>>
>> Seems reasonable.
>
> Here's a patch implementing that. Docs not updated yet. I did not change
> the way checkpoint_segments triggers checkpoints - that'll can be a
> separate patch. This only decouples the segment preallocation behavior
> from checkpoint_segments. With the patch, you can set
> checkpoint_segments really high, without consuming that much disk space
> all the time.

I don't understand what this patch, by itself, will accomplish in terms
of the originally stated goals of making checkpoint_segments easier to
tune, and controlling disk space used. To some degree, it makes both of
these things worse, because you can no longer use checkpoint_segments to
control the disk space. Instead, it is replaced by magic.

What sort of behavior are you expecting to come out of this? In testing,
I didn't see much of a difference. Although I'd expect that this would
actually preallocate fewer segments than the old formula.

Two small issues in the code:

Code change doesn't match comment:

+ *
+ * XXX: We don't have a good estimate of how many WAL files we should keep
+ * preallocated here. Quite arbitrarily, use max_advance=5. That's good
+ * enough for current use of this function; this only gets called when
+ * there are no more preallocated WAL segments available.
*/
installed_segno = logsegno;
- max_advance = XLOGfileslop;
+ max_advance = CheckPointSegments;

KB should be kB.