Re: PATCH: regular logging of checkpoint progress

On 2 Září 2011, 22:04, Tom Lane wrote:
> "Tomas Vondra" <tv(at)fuzzy(dot)cz> writes:
>> On 2 Z?????? 2011, 21:23, Tom Lane wrote:
>>> Well, to be blunt, putting stuff into the postmaster log is entirely
>>> the
>>> wrong way to satify a requirement like that. If you want to expose
>>> progress information, it should be exposed via something dynamic like
>>> pg_stat_activity. What could be useful to log is statistics that
>>> people
>>> might want to aggregate later, and I don't immediately see a reason why
>>> such stats couldn't be logged just once at end of each checkpoint
>>> cycle.
>
>> The problem with pg_stat_activity is that it provides just 'current
>> state', no history. If you don't sample that often enough, you may
>> completely miss the checkpoint (and thus you'll have no info about it,
>> unless you enable log_checkpoints and check the log). And it's imposible
>> to use if you need info about something that happened in the past. And
>> AFAIK it does not show processes running timed checkpoints for example.
>
> Your requirements seem sufficiently slippery that I don't think you've
> thought them through very well. As far as I can see, the proposed patch
> will mostly result in bloating the postmaster log with repetitive
> information of next to no value. I can see the reason for wanting to
> know what the system is doing right now, and I can see the reason for
> wanting aggregatable statistics so that you can tell over time whether
> your settings need to be adjusted. I don't see the value in a lot of
> "10% done" log entries --- there is essentially no historical value in
> such, IMO, because they don't correspond to any user-level activity.
> (Which is what distinguishes this from, say, log_connections or
> log_statements.) The fact that you can't figure out a reasonable
> frequency for making the entries is a symptom of the design being wrong
> at its core.

No, I probably haven't thought through all the requirements - but that's
exactly the feedback I expect from a commit fest.

Originally I've built the patch to fix a single use-case, where the 'log
each 10%' approach made perfect sense. I'm the first one to admin this is
not a good approach for the other use-cases and that the frequency should
be figured out in a different way.

Anyway I don't think that a piece of information that does not correspond
to any user-level activity is useless. Checkpoints are often a very
intrusively and significantly influence the users - that's why I believe
any help to tune them is desirable. The ability to see what the system is
doing right now and aggregatable statistics are fine, but in many cases
they're useless (e.g. when investigating past events).

Tomas