Re: Identify missing publications from publisher while create/alter subscription.

On Thu, Mar 4, 2021 at 1:04 PM Bharath Rupireddy
<bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
>
> On Wed, Mar 3, 2021 at 8:59 AM Euler Taveira <euler(at)eulerto(dot)com> wrote:
> >
> > On Wed, Feb 3, 2021, at 2:13 AM, Bharath Rupireddy wrote:
> >
> > On Mon, Jan 25, 2021 at 10:32 PM vignesh C <vignesh21(at)gmail(dot)com> wrote:
> >
> > > If a publication which does not exist is specified during create subscription, then we should throw an error similar to step 2 behavior. Similarly if a publication which does not exist is specified during alter subscription, then we should throw an error similar to step 4 behavior. If publication is dropped after subscription is created, this should be removed when an alter subscription subname refresh publication is performed similar to step 4.
> > > Thoughts?
> >
> > Postgres implemented a replication mechanism that is decoupled which means that
> > both parties can perform "uncoordinated" actions. As you shown, the CREATE
> > SUBSCRIPTION informing a non-existent publication is one of them. I think that
> > being permissive in some situations can prevent you from painting yourself into
> > a corner. Even if you try to be strict on the subscriber side, the other side
> > (publisher) can impose you additional complexity.
> >
> > You are arguing that in the initial phase, the CREATE SUBSCRIPTION has a strict
> > mechanism. It is a fair point. However, impose this strictness for the other
> > SUBSCRIPTION commands should be carefully forethought. If we go that route, I
> > suggest to have the current behavior as an option. The reasons are: (i) it is
> > backward-compatible, (ii) it allows us some known flexibility (non-existent
> > publication), and (iii) it would probably allow us to fix a scenario created by
> > the strict mode. This strict mode can be implemented via new parameter
> > validate_publication (ENOCAFFEINE to propose a better name) that checks if the
> > publication is available when you executed the CREATE SUBSCRIPTION. Similar
> > parameter can be used in ALTER SUBSCRIPTION ... SET PUBLICATION and ALTER
> > SUBSCRIPTION ... REFRESH PUBLICATION.
>
> IIUC the validate_publication is kind of the feature enable/disable
> flag. With the default value being false, when set to true, it checks
> whether the publications exists or not while CREATE/ALTER SUBSCRIPTION
> and throw error. If I'm right, then the validate_publication option
> looks good to me. So, whoever wants to impose strict restrictions to
> CREATE/ALTER subscription can set it to true. All others will not see
> any new behaviour or such.
>
> > To not mandate any new behaviour, I would suggest to have a new option
> > for ALTER SUBSCRIPTION ... REFRESH PUBLICATION WITH
> > (remove_dropped_publications = false). The new option
> > remove_dropped_publications will have a default value false, when set
> > to true it will check if the publications that are present in the
> > subscription list are actually existing on the publisher or not, if
> > not remove them from the list. And also in the documentation we need
> > to clearly mention the consequence of this new option setting to true,
> > that is, the dropped publications if created again will need to be
> > added to the subscription list again.
> >
> > REFRESH PUBLICATION is not the right command to remove publications. There is a
> > command for it: ALTER SUBSCRIPTION ... SET PUBLICATION.
> >
> > The other alternative is to document that non-existent publication names can be
> > in the subscription catalog and it is ignored while executing SUBSCRIPTION
> > commands. You could possibly propose a NOTICE/WARNING that informs the user
> > that the SUBSCRIPTION command contains non-existent publication.
>
> I think, we can also have validate_publication option allowed for
> ALTER SUBSCRIPTION SET PUBLICATION and REFRESH PUBLICATION commands
> with the same behaviour i.e. error out when specified publications
> don't exist in the publisher. Thoughts?

Sorry for the delayed reply, I was working on a few other projects so
I was not able to reply quickly.
Since we are getting the opinion that if we make the check
publications by default it might affect the existing users, I'm fine
with having an option validate_option to check if the publication
exists in the publisher, that way there is no change for the existing
user. I have made a patch in similar lines, attached patch has the
changes for the same.
Thoughts?

Regards,
Vignesh