Re: Improve documentation for pg_upgrade, standbys and rsync

Greetings,

* Robert Haas (robertmhaas(at)gmail(dot)com) wrote:
> On Mon, Jul 26, 2021 at 3:11 PM Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> > * Laurenz Albe (laurenz(dot)albe(at)cybertec(dot)at) wrote:
> > > Thanks for looking at this!
> >
> > Sure. Thanks for working on it!
>
> Stephen, do you intend to do something about this patch in terms of
> getting it committed? You're the only reviewer but haven't responded
> to the thread for more than 5 months.

I tried to be clear in the last email on the thread, the one which you
just responded to, here:

* Stephen Frost (sfrost(at)snowman(dot)net) wrote:
> This, of course, all comes back to the original complaint I had about
> documenting this approach, which is that these things should only be
> done by someone extremely familiar with the PG codebase, until and
> unless we write an actual tool to do this.

To be more explicit though- we should write a tool to do this. We
shouldn't try to document a way to do it because it's hard to get right.
While rsync is very capable, what's needed to really do this goes beyond
what we could reasonably put into any rsync command or really even into
a documented procedure. I get that we already have it documented (and
I'll note that doing so was against my recommendation..) and that some
folks (likely those who follow this mailing list) have had success using
it, but I'd really rather we just take it out and put it on a wiki
somewhere as a "we need a tool that does this stuff" and hope that
someone finds time to write one.

> I don't feel that I know this area of the documentation well enough to
> feel comfortable passing judgement on whether this change is an
> improvement or not. However I do feel somewhat uncomfortable with
> this:
>
> - <step>
> - <title>Prepare for standby server upgrades</title>
> -
> - <para>
> - If you are upgrading standby servers using methods outlined in
> section <xref
> - linkend="pgupgrade-step-replicas"/>, verify that the old standby
> - servers are caught up by running <application>pg_controldata</application>
> - against the old primary and standby clusters. Verify that the
> - <quote>Latest checkpoint location</quote> values match in all clusters.
> - (There will be a mismatch if old standby servers were shut down
> - before the old primary or if the old standby servers are still running.)
> - Also, make sure <varname>wal_level</varname> is not set to
> - <literal>minimal</literal> in the
> <filename>postgresql.conf</filename> file on the
> - new primary cluster.
> - </para>
> - </step>
>
> Right now, we say that you should stop the standby servers and then
> prepared for standby server upgrades. With this patch, we say that you
> should first prepare for standby server upgrades, and then stop the
> standby servers. But the last part of the text about preparing for
> standby server upgrades now mentions things to be done after carrying
> out the next step where the servers are actually stopped. That seems
> confusing. Perhaps we need two separate steps here, one to be
> performed before stopping both servers and the other after.

It should really be both- things to do on the primary ahead of time
(truncate all unlogged tables, make sure there aren't any orphaned
temporary tables, etc), and then things to do on the replicas after
shutting the primary down (basically, make sure they are fully caught up
with where the primary was at shutdown). I tried to explain that in my
prior email but perhaps didn't do a very good job.

> Also, let me express my general terror at the idea of anyone actually
> using this procedure.

I mean, yeah, I agree.

Thanks,

Stephen