Re: speed up a logical replica setup

On Thu, Mar 21, 2024 at 8:00 PM Euler Taveira <euler(at)eulerto(dot)com> wrote:
>
> On Thu, Mar 21, 2024, at 10:33 AM, vignesh C wrote:
>
> If we commit this we might not be able to change the way the option
> behaves once the customers starts using it. How about removing these
> options in the first version and adding it in the next version after
> more discussion.
>
>
> We don't need to redesign this one if we want to add a format string in a next
> version. A long time ago, pg_dump started to accept pattern for tables without
> breaking or deprecating the -t option. If you have 100 databases and you don't
> want to specify the options or use a script to generate it for you, you also
> have the option to let pg_createsubscriber generate the object names for you.
> Per my experience, it will be a rare case.
>

But, why go with some UI in the first place which we don't think is a
good one, or at least don't have a broader agreement that it is a good
one? The problem with self-generated names for users could be that
they won't be able to make much out of it. If one has to always use
those internally then probably that would be acceptable. I would
prefer what Tomas proposed a few emails ago as compared to this one as
that has fewer options to be provided by users but still, they can
later identify objects. But surely, we should discuss if you or others
have better alternatives.

The user choosing to convert a physical standby to a subscriber would
in some cases be interested in converting it for all the databases
(say for the case of upgrade [1]) and giving options for each database
would be cumbersome for her.

> Currently dry-run will do the check and might fail on identifying a
> few failures like after checking subscriber configurations. Then the
> user will have to correct the configuration and re-run then fix the
> next set of failures. Whereas the suggest-config will display all the
> optimal configuration for both the primary and the standby in a single
> shot. This is not a must in the first version, it can be done as a
> subsequent enhancement.
>
>
> Do you meant publisher, right? Per order, check_subscriber is done before
> check_publisher and it checks all settings on the subscriber before exiting. In
> v30, I changed the way it provides the required settings. In a previous version,
> it fails when it found a wrong setting; the current version, check all settings
> from that server before providing a suitable error.
>
> pg_createsubscriber: checking settings on publisher
> pg_createsubscriber: primary has replication slot "physical_slot"
> pg_createsubscriber: error: publisher requires wal_level >= logical
> pg_createsubscriber: error: publisher requires 2 replication slots, but only 0 remain
> pg_createsubscriber: hint: Consider increasing max_replication_slots to at least 3.
> pg_createsubscriber: error: publisher requires 2 wal sender processes, but only 0 remain
> pg_createsubscriber: hint: Consider increasing max_wal_senders to at least 3.
>
> If you have such an error, you will fix them all and rerun using dry run mode
> again to verify everything is ok. I don't have a strong preference about it. It
> can be changed easily (unifying the check functions or providing a return for
> each of the check functions).
>

We can unify the checks but not sure if it is worth it. I am fine
either way. It would have been better if we provided a way for a user
to know the tool's requirement rather than letting him know via errors
but I think it should be okay to extend it later as well.

[1] - This tool can be used in an upgrade where the user first
converts physical standby to subscriber to get incremental changes and
then performs an online upgrade.

--
With Regards,
Amit Kapila.