Re: PATCH: regular logging of checkpoint progress

On 26 Srpen 2011, 19:17, Greg Smith wrote:
> On 08/26/2011 03:54 AM, Magnus Hagander wrote:
>> In theory, this could be the "progress view" or
>> "progress field" talked about around Gregs previous patch - or it
>> could just be modifying the commandstring in pg_stat_activity.
>
> Right. The whole progress indicator idea is hard to do for queries in
> general. But there's enough of these other progress indicator ideas
> around now that it may be worth putting a standard way to handle them in
> here. It sounds like that would be sufficient to address the area Tomas
> is trying to instrument better. I badly want a progress indicator on
> CREATE INDEX CONCURRENTLY too, to at least let me know what phase of the
> build process it's on. That's turned into a major headache recently.
>
> If we run with the idea of just allowing backends to publish a progress
> text string, I think this one maps into a similar space as the
> autovacuum one. Publishing how many seconds the operation has been
> running for may be reasonable too. Whether the overhead of the timing
> calls necessary to compute that will be high or not depends on the
> refresh rate of the progress info. My suggestion before was to name
> these as key=value pairs for easy parsing; here's three examples now:
>
> autovacumm: pgbench_accounts h=182701 m=301515 d=321345 s=62.231
> (cache hits, cache misses, dirty writes, seconds)
>
> background writer: checkpoint b=511 t=3072 s=5.321
> (buffers written, total, seconds)
>
> create index concurrently: pgbench_accounts p=1 b=62 t=6213 s=81.232
> (phase, blocks processed, total block estimate, seconds)
>
> I think that the idea of making this easily human readable is
> optimistic, because it will make all these strings big enough to start
> mattering. Given that, we almost have to assume the only consumers of
> this data will be able to interpret it using the documentation. I'd be
> happy with just the minimal data set in each case, not including any
> statistics you can easily derive from the values given (like the MB/s
> readings). Adding that figure in particular to more of the log messages
> would be nice though.

I'm a bit confused - are you talking about updating process title or about
writing the info to log? The process title is probably fine for watching
the progress interactively, but it really does not solve what I need.

I need to be able to infer the progress for past events, so I'd have to
sample the 'ps ax' output regularly.

Tomas