Re: Add REPACK progress phases for logical decoding setup

Hi Sami, Masahiko,

I tried this patch with the PROGRESS_DEBUG trace and test framework I
posted earlier today in a different thread [1], which logs every change
to a backend's progress state and checks it against rules that hold for
every command.

It applies cleanly on top of today's master, no warnings. With the
option, make check (239 tests) and the framework's own checks pass with
no violations, across 1.34 million logged progress changes. For REPACK
(CONCURRENTLY) the new phases show up in the order the patch documents,
9 -> 10 -> 11, and never for a plain REPACK, matching "skipped when not
in concurrent mode". I also ran a plain REPACK (CONCURRENTLY) 20 times
with wal_level already set to logical, the case the commit message calls
"effectively instantaneous": phase 9 was still captured every time, so
the backend's polling loop isn't racing past it in practice, at least
not in this setup.

Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> While it's true that the logical decoding might need to read WAL
> records until the snapshot reaches a consistent state, I'm not sure
> that it can justify having its own phase. I think we can merge it to
> (2) and have something like a "initializing logical decoding" phase.

I tried to find a case where building the initial snapshot (phase 11)
takes meaningfully long on its own, independent of the old-transaction
wait (phase 10), since that would be the argument for keeping them
separate. Timing the phase changes from the trace:

- An open transaction holding an XID for 3s, no other activity:
phase 10 measured 2794 ms, phase 11 measured 2 ms.

- No open transaction, but a few hundred autocommitted inserts on an
unrelated table running while REPACK starts, to put some WAL in the
way: phase 10 measured 0 ms, phase 11 measured 1 ms.

In both cases, and in everything else I tried, phase 11 stayed under a
few milliseconds, and whatever was slow showed up in phase 10 instead.
I could not produce a case where phase 11 alone was slow. That lines
up with what you suspected, though a small, short-lived test like this
one obviously does not rule out phase 11 mattering under conditions I
did not try, such as a very large number of subtransactions or a
snapshot builder that is already far behind for unrelated reasons.

[1] https://postgr.es/m/CA+bCEdBKvmoOd=ShLZA99FNHFOc5kjdPRzfOZLgSdcm07uy28g@mail.gmail.com

Regards,
Manu