RE: Time delayed LR (WAS Re: logical replication restrictions)

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>, "Takamichi Osumi (Fujitsu)" <osumi(dot)takamichi(at)fujitsu(dot)com>
Cc: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "vignesh21(at)gmail(dot)com" <vignesh21(at)gmail(dot)com>, "euler(at)eulerto(dot)com" <euler(at)eulerto(dot)com>, "m(dot)melihmutlu(at)gmail(dot)com" <m(dot)melihmutlu(at)gmail(dot)com>, "andres(at)anarazel(dot)de" <andres(at)anarazel(dot)de>, "marcos(at)f10(dot)com(dot)br" <marcos(at)f10(dot)com(dot)br>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, "smithpb2250(at)gmail(dot)com" <smithpb2250(at)gmail(dot)com>
Subject: RE: Time delayed LR (WAS Re: logical replication restrictions)
Date: 2022-12-16 06:41:03
Message-ID: TYAPR01MB5866DE4CC2D3D5EE15973A9DF5E69@TYAPR01MB5866.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Amit,

> I also don't see the need for this mechanism for logical replication,
> and in fact, why do we need to even wait for sending the existing WAL?

Is it meant that logicalrep walsenders do not have to track WalSndCaughtUp and
any pending data in the output buffer?

> I think the reason why we don't need to wait for logical replication
> is that after the restart, we always start sending WAL from the
> location requested by the subscriber, or till the point where the
> publisher knows the confirmed flush location of the subscriber.
> Consider another case where after restart publisher (node-1) wants to
> act as a subscriber for the previous subscriber (node-2). Now, the new
> subscriber (node-1) won't have a way to tell the new publisher
> (node-2) that starts from the location where the node-1 went down as
> WAL locations between publisher and subscriber need not be same.

You mean to say that such mechanism was made for supporting switchover, but logical
replication cannot do because new subscriber cannot request definitively unknown
changes for it, right? It seems reasonable to me.

> This brings us to the question of whether users can use logical
> replication for the scenario where they want the old master to follow
> the new master after the restart which we typically do in physical
> replication, if so how?

Maybe to support such use-case, 2-way replication is needed
(but this is out-of-scope of this thread).

> Another related point to consider is what is the behavior of
> synchronous replication when shutdown has been performed both in the
> case of physical and logical replication especially when the
> time-delayed replication feature is enabled?

In physical replication without any failures, it seems that users can stop primary
server even if the applications are delaying on secondary. This is because sent WALs
are immediately flushed on secondary and walreceiver replies its position. The
transaction has been already committed at that time, and the transported changes
will be applied on secondary after spending time.

IIUC we can achieve that when logical walsenders do not consider the remote status
while shutting down, but I want to hear another opinion and we must confirm by testing...

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrey Lepikhov 2022-12-16 06:45:30 Re: Removing unneeded self joins
Previous Message Japin Li 2022-12-16 05:21:59 Re: Typo macro name on FreeBSD?