RE: Synchronizing slots from primary to standby

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Cc: "Drouvot, Bertrand" <bertranddrouvot(dot)pg(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Ajin Cherian <itsajin(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Subject: RE: Synchronizing slots from primary to standby
Date: 2023-11-02 09:04:57
Message-ID: OS0PR01MB57168A5C52E69FCE8A84DFC794A6A@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thursday, November 2, 2023 8:27 AM Zhijie Hou (Fujitsu) <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Tuesday, October 31, 2023 6:45 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> >
> > On Fri, Oct 27, 2023 at 4:04 PM shveta malik <shveta(dot)malik(at)gmail(dot)com>
> wrote:
> > >
> > > On Fri, Oct 27, 2023 at 3:26 PM shveta malik
> > > <shveta(dot)malik(at)gmail(dot)com>
> > wrote:
> > > ==========
> > >
> > > Open questions regarding change for pt 1 above:
> > > a) I think we should restrict the 'alter-sub set failover' when
> > > failover-state is currently in 'p' (pending) state i.e. table-sync
> > > is going over. Once table-sync is over, then toggle of 'failover'
> > > should be allowed using alter-subscription.
> > >
> >
> > Agreed.
> >
> > > b) Currently I have restricted 'alter subscription.. refresh
> > > publication with copy=true' when failover=true (on a similar line of
> > > two-phase). The reason being, refresh with copy=true will go for
> > > table-sync again and since failover was set in main-slot after
> > > table-sync was done, it will need going through the same transition
> > > of 'p' to 'e' for main slot making it unsyncable for that time.
> > > Should it be allowed?
> > >
> >
> > Yeah, I also think we can't allow refresh with copy=true when
> > 'failover' is enabled.
> >
> > I think the current implementation of this flag seems a bit clumsy
> > because 'failover' is a slot property and we are trying to map it to
> > plugin_options. It has to be considered similar to the opt_temporary option
> while creating the slot.
> >
> > We have create_replication_slot and drop_replication_slot in
> > repl_gram.y. How about if introduce alter_replication_slot and handle the
> 'failover' flag with that?
> > The idea is we will either enable 'failover' at the time
> > create_replication_slot by providing an optional failover option or
> > execute a separate command alter_replication_slot. I think we probably
> > need to perform this command before the start of streaming.
>
> Here is an attempt to achieve the same. I added a new replication command
> alter_replication_slot and introduced a walreceiver api walrcv_alter_slot to
> execute the command. The subscription will call the api to enable/disable the
> failover of the slot on publisher.

Here is the new version patch set(V29) which addressed Peter comments[1][2] and
fixed one doc compile error.

Thanks Ajin for helping address some of the comments.

[1] https://www.postgresql.org/message-id/CAHut%2BPspseC03Fhsi%3DOqOtksagspE%2B0MVOhrhhUb64cc_4SE1w%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAHut%2BPubYbmLpGeOd2QTBPhHwtZa-Qm9Kg38Cu_EiG%2B1RbV47g%40mail.gmail.com

Best Regards,
Hou zj

Attachment Content-Type Size
v29-0001-Allow-logical-walsenders-to-wait-for-the-physica.patch application/octet-stream 127.4 KB
v29-0002-Add-logical-slot-sync-capability-to-the-physical.patch application/octet-stream 116.2 KB
v29-0003-Allow-slot-sync-workers-to-wait-for-the-cascadin.patch application/octet-stream 7.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2023-11-02 09:06:09 Re: A recent message added to pg_upgade
Previous Message Shlok Kyal 2023-11-02 09:03:40 Re: psql not responding to SIGINT upon db reconnection