Re: Synchronizing slots from primary to standby

From: shveta malik <shveta(dot)malik(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, "Drouvot, Bertrand" <bertranddrouvot(dot)pg(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Ajin Cherian <itsajin(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: Re: Synchronizing slots from primary to standby
Date: 2023-09-19 04:50:55
Message-ID: CAJpy0uDyqBdssDbPjvjDtG5HfmiMtKZrGty4r-E-0jh6e9v7OQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Sep 13, 2023 at 5:19 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Sep 13, 2023 at 4:54 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
> >
> > PFA v17. It has below changes:
> >
>
> @@ -2498,6 +2500,13 @@ ReorderBufferProcessTXN(ReorderBuffer *rb,
> ReorderBufferTXN *txn,
> }
> else
> {
> + /*
> + * Before we send out the last set of changes to logical decoding
> + * output plugin, wait for specified streaming replication standby
> + * servers (if any) to confirm receipt of WAL upto commit_lsn.
> + */
> + WaitForStandbyLSN(commit_lsn);
>
> It seems the first patch has a wait logic for every commit. I think it
> is better to integrate this wait with WalSndWaitForWal() as suggested
> by Andres in his email[1].
>
> [1] - https://www.postgresql.org/message-id/20220207204557.74mgbhowydjco4mh%40alap3.anarazel.de
>
> --

Sure Amit. PFA v18. It addresses below:

1) patch001: wait for physical-standby confirmation logic is now
integrated with WalSndWaitForWal(). Now walsender waits for physical
standby's confirmation to take changes upto RecentFlushPtr in
WalSndWaitForWal(). This allows walsender to send the changes to
logical subscribers one by one which are already covered in
RecentFlushPtr without needing to wait on every commit for physical
standby confirmation.

2) if synchronize_slot_names set on physical standby has physical slot
name, primary's walsender on receiving that will error out. This is
currently done in ListSlotDatabaseOIDs(), but it needs to be moved to
logic where standby will send synchronize_slot_names to be set on
primary and primary will validate that first. GUC
synchronize_slot_names will be removed from primary. This arrangement
to be done in next version.

3) Peter's comment dated Sep15.

thanks
Shveta

Attachment Content-Type Size
v18-0002-Add-logical-slot-sync-capability-to-physical-sta.patch application/octet-stream 103.0 KB
v18-0001-Allow-logical-walsenders-to-wait-for-physical-st.patch application/octet-stream 25.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message shveta malik 2023-09-19 04:59:26 Re: Synchronizing slots from primary to standby
Previous Message Andrey Lepikhov 2023-09-19 04:42:13 Re: POC: GROUP BY optimization