Re: PATCH: regular logging of checkpoint progress

On 08/26/2011 03:54 AM, Magnus Hagander wrote:
> In theory, this could be the "progress view" or
> "progress field" talked about around Gregs previous patch - or it
> could just be modifying the commandstring in pg_stat_activity.

Right. The whole progress indicator idea is hard to do for queries in
general. But there's enough of these other progress indicator ideas
around now that it may be worth putting a standard way to handle them in
here. It sounds like that would be sufficient to address the area Tomas
is trying to instrument better. I badly want a progress indicator on
CREATE INDEX CONCURRENTLY too, to at least let me know what phase of the
build process it's on. That's turned into a major headache recently.

If we run with the idea of just allowing backends to publish a progress
text string, I think this one maps into a similar space as the
autovacuum one. Publishing how many seconds the operation has been
running for may be reasonable too. Whether the overhead of the timing
calls necessary to compute that will be high or not depends on the
refresh rate of the progress info. My suggestion before was to name
these as key=value pairs for easy parsing; here's three examples now:

autovacumm: pgbench_accounts h=182701 m=301515 d=321345 s=62.231
(cache hits, cache misses, dirty writes, seconds)

background writer: checkpoint b=511 t=3072 s=5.321
(buffers written, total, seconds)

create index concurrently: pgbench_accounts p=1 b=62 t=6213 s=81.232
(phase, blocks processed, total block estimate, seconds)

I think that the idea of making this easily human readable is
optimistic, because it will make all these strings big enough to start
mattering. Given that, we almost have to assume the only consumers of
this data will be able to interpret it using the documentation. I'd be
happy with just the minimal data set in each case, not including any
statistics you can easily derive from the values given (like the MB/s
readings). Adding that figure in particular to more of the log messages
would be nice though.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us