RE: Time delayed LR (WAS Re: logical replication restrictions)

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Masahiko Sawada' <sawada(dot)mshk(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "andres(at)anarazel(dot)de" <andres(at)anarazel(dot)de>, "vignesh21(at)gmail(dot)com" <vignesh21(at)gmail(dot)com>, "shveta(dot)malik(at)gmail(dot)com" <shveta(dot)malik(at)gmail(dot)com>, "Takamichi Osumi (Fujitsu)" <osumi(dot)takamichi(at)fujitsu(dot)com>, "dilipbalaut(at)gmail(dot)com" <dilipbalaut(at)gmail(dot)com>, "euler(at)eulerto(dot)com" <euler(at)eulerto(dot)com>, "m(dot)melihmutlu(at)gmail(dot)com" <m(dot)melihmutlu(at)gmail(dot)com>, "marcos(at)f10(dot)com(dot)br" <marcos(at)f10(dot)com(dot)br>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: Time delayed LR (WAS Re: logical replication restrictions)
Date: 2023-03-01 09:21:11
Message-ID: TYAPR01MB5866085A0FD56488303B99D4F5AD9@TYAPR01MB5866.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Sawada-san,

Thank you for giving your consideration!

> > We have documented at least one such case
> > already where during Drop Subscription, if the network is not
> > reachable then also, a similar problem can happen and users need to be
> > careful about it [1].
>
> Apart from a bad-use case example I mentioned, in general, piling up
> WAL files due to the replication slot has many bad effects on the
> system. I'm concerned that the side effect of this feature (at least
> of the current design) is too huge compared to the benefit, and afraid
> that users might end up using this feature without understanding the
> side effect well. It might be okay if we thoroughly document it but
> I'm not sure.

One approach is that change max_slot_wal_keep_size forcibly when min_send_delay
is set. But it may lead to disable the slot because WALs needed by the time-delayed
replication may be also removed. Just the right value cannot be set by us because
it is quite depends on the min_send_delay and workload.

How about throwing the WARNING when min_send_delay > 0 but
max_slot_wal_keep_size < 0? Differ from previous, version the subscription
parameter min_send_delay will be sent to publisher. Therefore, we can compare
min_send_delay and max_slot_wal_keep_size when publisher receives the parameter.

Of course we can reject such a setup by using ereport(ERROR), but it may generate
abandoned replication slot. It is because we send the parameter at START_REPLICATION
and the slot has been already created.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jim Jones 2023-03-01 09:29:06 Re: Proposal: %T Prompt parameter for psql for current time (like Oracle has)
Previous Message Heikki Linnakangas 2023-03-01 09:12:35 Re: refactoring relation extension and BufferAlloc(), faster COPY