RE: Synchronizing slots from primary to standby

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>
Cc: shveta malik <shveta(dot)malik(at)gmail(dot)com>, Ajin Cherian <itsajin(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Subject: RE: Synchronizing slots from primary to standby
Date: 2024-01-25 13:11:50
Message-ID: OS0PR01MB571623B1D01003F99A7BB983947A2@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thursday, January 25, 2024 6:25 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Thu, Jan 25, 2024 at 1:25 PM Bertrand Drouvot
> <bertranddrouvot(dot)pg(at)gmail(dot)com> wrote:
> >
> > On Thu, Jan 25, 2024 at 02:57:30AM +0000, Zhijie Hou (Fujitsu) wrote:
> > >
> > > Agreed. I split the original 0001 patch into 3 patches as suggested.
> > > Here is the V68 patch set.
>
> Thanks, I have pushed 0001.
>
> >
> > Thanks!
> >
> > Some comments.
> >
> > Looking at 0002:
> >
> > 1 ===
> >
> > + <para>The following options are supported:</para>
> >
> > What about "The following option is supported"? (as currently only the
> "FAILOVER"
> > is)
> >
> > 2 ===
> >
> > What about adding some TAP tests too? (I can see that
> > ALTER_REPLICATION_SLOT test is added in v68-0004 but I think having some
> in 0002 would make sense too).
> >
>
> The subscription tests in v68-0003 will test this functionality. The one
> advantage of adding separate tests for this is that if in the future we extend this
> replication command, it could be convenient to test various options. However,
> the same could be said about existing replication commands as well. But is it
> worth having extra tests which will be anyway covered in the next commit in a
> few days?
>
> I understand that it is a good idea and makes one comfortable to have tests for
> each separate commit but OTOH, in the longer term it will just be adding more
> test time without achieving much benefit. I think we can tell explicitly in the
> commit message of this patch that the subsequent commit will cover the tests
> for this functionality

Agreed.

>
> One minor comment on 0002:
> + so that logical replication can be resumed after failover.
> + </para>
>
> Can we move this and similar comments or doc changes to the later 0004 patch
> where we are syncing the slots?

Thanks for the comment.

Here is the V69 patch set which includes the following changes.

V69-0001, V69-0002
1) Addressed Bertrand's comments[1].

V69-0003
1) Addressed Peter's comment in [2], [3]
2) Addressed Amit's comment in [4] and above.
3) Fixed one issue that the startup process may report ERROR if it tries to drop
the same slot that the slotsync worker is acquiring. Now we take shared lock on
db in slot-sync worker before we create, update or drop any of its slots. This
is done to prevent potential conflict with ReplicationSlotsDropDBSlots() in
case that database is dropped in parallel.

V69-0004
1) Rebased and fixed one CFbot failure.

V69-0005, V69-0006, V69-0007
1) Rebased.

Thanks Shveta for rebasing and working for the changes on 0003~0007.

[1] https://www.postgresql.org/message-id/ZbIT9Kj3d8TFD8h6%40ip-10-97-1-34.eu-west-3.compute.internal
[2]: https://www.postgresql.org/message-id/CAHut%2BPt2oLfxv_%3DGN23dOOduKHBHdAkCvwSZiwSbtTJFFbQm-w%40mail.gmail.com
[3]: https://www.postgresql.org/message-id/CAHut%2BPtsDYPbg7qM1nGWtJcSQBQ5JH%3DLmgyqwqBPL9k%2Bz8f5Ew%40mail.gmail.com
[4]: https://www.postgresql.org/message-id/CAA4eK1%2B4PhO-f4%2B2fForG6MOEj3jbtee_PYPtwtgww%3DonC5DSQ%40mail.gmail.com

Best Regards,
Hou zj

Attachment Content-Type Size
v69-0006-Non-replication-connection-and-app_name-change.patch application/octet-stream 9.7 KB
v69-0007-Document-the-steps-to-check-if-the-standby-is-re.patch application/octet-stream 6.6 KB
v69-0003-Add-logical-slot-sync-capability-to-the-physical.patch application/octet-stream 91.4 KB
v69-0004-Slot-sync-worker-as-a-special-process.patch application/octet-stream 38.4 KB
v69-0005-Allow-logical-walsenders-to-wait-for-the-physica.patch application/octet-stream 41.2 KB
v69-0002-Add-a-failover-option-to-subscriptions.patch application/octet-stream 63.8 KB
v69-0001-Allow-setting-failover-property-in-the-replicati.patch application/octet-stream 19.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Aleksander Alekseev 2024-01-25 13:16:55 Re: Remove unused fields in ReorderBufferTupleBuf
Previous Message David Steele 2024-01-25 12:56:52 Re: Use of backup_label not noted in log