Re: Load Distributed Checkpoints, take 3

On Fri, 22 Jun 2007, Tom Lane wrote:

> Yeah, I'm not sure that we've thought through the interactions with the
> existing bgwriter behavior.

The entire background writer mess needs a rewrite, and the best way to
handle that is going to shift considerably with LDC applied.

As the person who was complaining about corner cases I'm not in a position
to talk more explicitly about, I can at least summarize my opinion of how
I feel everyone should be thinking about this patch and you can take what
you want from that. In the context of getting an 8.3 release finalized, I
think you should be building a LDC implementation that accomplishes the
main goal of spreading the checkpoints out, which is clearly working, and
erring on the side of more knobs in cases where it's unsure if they're
needed or not. It's clear what my position is which non-obvious knobs I
think are important for some people.

Which is worse: putting in a tuning setting that it's discovered
everybody just uses the same value for, or discovering after release that
there's a use-case for that setting and by hard-coding it you've made the
DBAs for a class of applications you didn't think about miserable? In the
cases where there's good evidence so far of the right setting, just make
that the default, and the only harm is GUC bloat.

Nothing should be done that changes the existing behavior if the LDC
feature is turned off, so anything more obtrusive to the background writer
is right out. Make reducing the knobs, optimizing the default behavior,
and rewriting the background writer to better fit into its new context a
major goal of 8.4. I know I've got a whole notebook full of stuff on that
topic I've been ignoring as not to distract you guys from getting 8.3
done.

That's the low risk plan, and the design/beta/release period here is short
enough that I think going too experimental beyond that is a bad idea. To
pick an example, when I read this idea from Heikki:

> You would have a checkpoint running practically all the time, and you
> would use checkpoint_timeout/checkpoint_segments to control how long it
> takes... If we do that, we should remove bgwriter_all_* settings

Whether or not I think this is an awesome idea, the very idea of a change
that big at this point gives me the willies. Just off the top of my head,
there's a whole class of issues involving recycling xlog segments this
would introduce I would be really unhappy with the implications of. Did
anyone else ever notice that when a new xlog segment is created, the write
to clear it out doesn't happen via direct I/O like the rest of the xlog
writes do? That write goes through the regular buffered path instead.
The checkpoint logging patch I submitted logs when this happens
specifically because that particular issue messed with some operations and
I found it important to be aware of.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD