Re: Handle infinite recursion in logical replication setup

From: vignesh C <vignesh21(at)gmail(dot)com>
To: "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "kuroda(dot)hayato(at)fujitsu(dot)com" <kuroda(dot)hayato(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Handle infinite recursion in logical replication setup
Date: 2022-06-08 10:51:52
Message-ID: CALDaNm3Pt1CpEb3y9pE7ff91gZVpNXr91y4ZtWiw6h+GAyG4Gg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Jun 6, 2022 at 2:29 PM shiy(dot)fnst(at)fujitsu(dot)com
<shiy(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Mon, Jun 6, 2022 1:14 PM vignesh C <vignesh21(at)gmail(dot)com> wrote:
> >
> > The attached v18 patch has the fixes for the same.
> >
>
> Thanks for updating the patch, here are some comments.
>
> 0002 patch
> ==============
> 1.
> + <varlistentry>
> + <term><literal>origin</literal> (<type>string</type>)</term>
> + <listitem>
> + <para>
>
> It maybe better if the type of "origin" parameter is enum, as it cannot be any
> string and only has two valid values.

Currently we only support local and any. But this was designed so that
it can be extended to support origin names. Users can provide a
particular origin name to be filtered.
The same was also discussed in pg unconference as mentioned in [1]

> 2.
> @@ -607,6 +626,11 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
> LOGICALREP_TWOPHASE_STATE_PENDING :
> LOGICALREP_TWOPHASE_STATE_DISABLED);
> values[Anum_pg_subscription_subdisableonerr - 1] = BoolGetDatum(opts.disableonerr);
> + if (opts.origin)
> + values[Anum_pg_subscription_suborigin - 1] =
> + CStringGetTextDatum(opts.origin);
> + else
> + nulls[Anum_pg_subscription_suborigin - 1] = true;
> values[Anum_pg_subscription_subconninfo - 1] =
> CStringGetTextDatum(conninfo);
> if (opts.slot_name)
>
> Document of "CREATE SUBSCRIPTION" says, the default value of "origin" is "any", so why not set
> suborigin to "any" when user doesn't specify this parameter?

Modified

>
> 0003 patch
> ==============
> 1.
> @@ -300,6 +310,11 @@ CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceabl
> replication from the publisher. The default is
> <literal>false</literal>.
> </para>
> + <para>
> + There is some interaction between the <literal>origin</literal>
> + parameter and <literal>copy_data</literal> parameter. Refer to the
> + <xref linkend="sql-createsubscription-notes" /> for details.
> + </para>
> </listitem>
> </varlistentry>
>
> I think this change should be put together with "origin" parameter, instead of
> "disable_on_error".

Modified

>
> 0004 patch
> ==============
> 1.
> + <para>
> + Now the bidirectional logical replication setup is complete between
> + <literal>node1</literal>, <literal>node2</literal> and
> + <literal>node2</literal>. Any subsequent changes in one node will
> + replicate the changes to the other nodes.
> + </para>
>
> I think "node1, node2 and node2" should be "node1, node2 and node3".

Modified

Thanks for the comments, the attached v19 patch has the changes for the same.
[1] - https://wiki.postgresql.org/wiki/PgCon_2022_Developer_Unconference#Logical_Replication_Origin_Filtering_and_Consistency

Regards,
Vignesh

Attachment Content-Type Size
v19-0001-Add-a-missing-test-to-verify-only-local-paramete.patch application/x-patch 4.3 KB
v19-0002-Skip-replication-of-non-local-data.patch application/x-patch 56.5 KB
v19-0004-Document-bidirectional-logical-replication-steps.patch application/x-patch 13.6 KB
v19-0003-Check-and-throw-an-error-if-publisher-tables-wer.patch application/x-patch 38.0 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2022-06-08 11:04:05 Re: Skipping schema changes in publication
Previous Message Robert Haas 2022-06-08 10:50:56 Re: Collation version tracking for macOS