Re: speed up a logical replica setup

On Thu, Nov 9, 2023, at 8:12 PM, Michael Paquier wrote:
> On Thu, Nov 09, 2023 at 03:41:53PM +0100, Peter Eisentraut wrote:
> > On 08.11.23 00:12, Michael Paquier wrote:
> >> - Should the subdirectory pg_basebackup be renamed into something more
> >> generic at this point? All these things are frontend tools that deal
> >> in some way with the replication protocol to do their work. Say
> >> a replication_tools?
> >
> > Seems like unnecessary churn. Nobody has complained about any of the other
> > tools in there.
>
> Not sure. We rename things across releases in the tree from time to
> time, and here that's straight-forward.

Based on this discussion it seems we have a consensus that this tool should be
in the pg_basebackup directory. (If/when we agree with the directory renaming,
it could be done in a separate patch.) Besides this move, the v3 provides a dry
run mode. It basically executes every routine but skip when should do
modifications. It is an useful option to check if you will be able to run it
without having issues with connectivity, permission, and existing objects
(replication slots, publications, subscriptions). Tests were slightly improved.
Messages were changed to *not* provide INFO messages by default and --verbose
provides INFO messages and --verbose --verbose also provides DEBUG messages. I
also refactored the connect_database() function into which the connection will
always use the logical replication mode. A bug was fixed in the transient
replication slot name. Ashutosh review [1] was included. The code was also indented.

There are a few suggestions from Ashutosh [2] that I will reply in another
email.

I'm still planning to work on the following points:

1. improve the cleanup routine to point out leftover objects if there is any
connection issue.
2. remove the physical replication slot if the standby is using one
(primary_slot_name).
3. provide instructions to promote the logical replica into primary, I mean,
stop the replication between the nodes and remove the replication setup
(publications, subscriptions, replication slots). Or even include another
action to do it. We could add both too.

Point 1 should be done. Points 2 and 3 aren't essential but will provide a nice
UI for users that would like to use it.

[1] https://postgr.es/m/CAExHW5sCAU3NvPKd7msScQKvrBN-x_AdDQD-ZYAwOxuWG%3Doz1w%40mail.gmail.com
[2] https://postgr.es/m/CAExHW5vHFemFvTUHe+7XWphVZJxrEXz5H3dD4UQi7CwmdMJQYg@mail.gmail.com

--
Euler Taveira
EDB https://www.enterprisedb.com/