Re: pg_stat_progress_create_index vs. parallel index builds

On 2021-Jun-02, Tomas Vondra wrote:

> On 6/2/21 6:38 PM, Alvaro Herrera wrote:

> > Hmm, that's odd. I distinctly recall testing the behavior with parallel
> > workers, and it is mentioned by Rahila in the original thread, and I
> > think we tried to ensure that it was sane. I am surprised to learn that
> > there's such a large gap.
>
> Yeah, I quickly skimmed [1] which I think is the thread you're referring
> to, and there is some discussion about parallel workers. I haven't read
> it in detail, though.
>
> [1]
> https://www.postgresql.org/message-id/20181220220022.mg63bhk26zdpvmcj%40alvherre.pgsql

Well, it is quite possible that we found *some* problems with parallel
workers but not all of them :-)

> > I think it would be valuable to have some kind of test mode where the
> > progress reporting APIs would make some noise (perhaps with a bespoke
> > GUC option) so that we can test things in some automated manner ...
>
> True, but how would that GUC work? Would it add something into the
> system view, or just log something?

With the GUC turned on, emit some sort of message (maybe at INFO level)
whenever some subset of the progress parameters changes. This makes it
easy to compare the progress of any command with the expected set of
messages.

However, it's not very clear which parameters are observed
for changes (you can't do it for all params, because you'd get one for
each block in some cases, and that's unworkable). The way have #defined
the parameters makes it difficult to annotate parameters with flag bits;
we could have something

#ifdef USE_ASSERT_CHECKING
#define PROGRESS_LOG_CHANGES 0x70000000
#else
#define PROGRESS_LOG_CHANGES 0x0
#endif
#define PROGRESS_CLUSTER_PHASE (1 | PROGRESS_LOG_CHANGES)

and the progress-reporting knows to mask-out the LOG_CHANGES bit before
storing the value in memory, but also knows to emit the log output if
that's enabled and the LOG_CHANGES bit is present. (The assertion flag
would be tested at compile time to avoid a performance hit in production
builds.)

--
Álvaro Herrera Valdivia, Chile
"I can't go to a restaurant and order food because I keep looking at the
fonts on the menu. Five minutes later I realize that it's also talking
about food" (Donald Knuth)