Re: Added schema level support for publication.

From: vignesh C <vignesh21(at)gmail(dot)com>
To: "tanghy(dot)fnst(at)fujitsu(dot)com" <tanghy(dot)fnst(at)fujitsu(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, Greg Nancarrow <gregn4422(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Peter Smith <smithpb2250(at)gmail(dot)com>, Ajin Cherian <itsajin(at)gmail(dot)com>, Rahila Syed <rahilasyed90(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
Subject: Re: Added schema level support for publication.
Date: 2021-09-30 10:09:07
Message-ID: CALDaNm2yJOEPCqR=gTMEwveJujH9c9_z4LhKmk2T3vZH7T1DLQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Sep 29, 2021 at 8:47 AM tanghy(dot)fnst(at)fujitsu(dot)com
<tanghy(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Monday, Tuesday, September 28, 2021 10:49 PM, vignesh C <vignesh21(at)gmail(dot)com> wrote:
> >
> > Yes this should not be supported, we should throw an error in this case.
> > This is handled in the v34 patch attached at [1].
> > [1] - https://www.postgresql.org/message-
> > id/CALDaNm2Z9TfuoCf09YGKfwy7F1NwC4iCXJGTaZS%3DchH6VHtadQ%40mail.g
> > mail.com
> >
>
> Thanks for fixing it. I confirmed the error can be output as expected.
>
> Here is a problem related to publish_via_partition_root option when using this
> patch. With this option on, I think pg_get_publication_tables function gave an
> unexcepted result and the subscriber would get dual data during table sync.
>
>
> For example:
> (I used pg_publication_tables view to make it looks clearer)
>
> create schema sch1;
> create table sch1.tbl1 (a int) partition by range ( a );
> create table sch1.tbl1_part1 partition of sch1.tbl1 for values from (1) to (10);
> create table sch1.tbl1_part2 partition of sch1.tbl1 for values from (10) to (20);
> create table sch1.tbl1_part3 partition of sch1.tbl1 for values from (20) to (30);
> create publication pub for all tables in schema sch1 with(publish_via_partition_root=1);
>
> postgres=# select * from pg_publication_tables where pubname='pub';
> pubname | schemaname | tablename
> ---------+------------+------------
> pub | sch1 | tbl1_part1
> pub | sch1 | tbl1_part2
> pub | sch1 | tbl1_part3
> pub | sch1 | tbl1
> (4 rows)
>
>
> It shows both the partitioned table and its leaf partitions. But the result of
> FOR ALL TABLES publication couldn't show the leaf partitions.
>
>
> postgres=# create publication pub_all for all tables with(publish_via_partition_root=1);
> CREATE PUBLICATION
> postgres=# select * from pg_publication_tables where pubname='pub_all';
> pubname | schemaname | tablename
> ---------+------------+-----------
> pub_all | sch1 | tbl1
> (1 row)
>
>
> How about make the following change to avoid it? I tried it and it also fixed dual
> data issue during table sync.
>
>
> diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
> index 04e785b192..4e8ccdabc6 100644
> --- a/src/backend/catalog/pg_publication.c
> +++ b/src/backend/catalog/pg_publication.c
> @@ -632,7 +632,8 @@ GetSchemaPublicationRelations(Oid schemaid, PublicationPartOpt pub_partopt)
> Form_pg_class relForm = (Form_pg_class) GETSTRUCT(tuple);
> Oid relid = relForm->oid;
>
> - if (is_publishable_class(relid, relForm))
> + if (is_publishable_class(relid, relForm) &&
> + !(relForm->relispartition && pub_partopt == PUBLICATION_PART_ROOT))
> result = lappend_oid(result, relid);
> }

The suggested change works, I have modified it in the attached patch.

Regards,
Vignesh

Attachment Content-Type Size
v35-0001-Added-schema-level-support-for-publication.patch text/x-patch 79.8 KB
v35-0002-Client-side-changes-to-support-FOR-ALL-TABLES-IN.patch text/x-patch 21.8 KB
v35-0003-Tests-for-FOR-ALL-TABLES-IN-SCHEMA-publication.patch text/x-patch 58.3 KB
v35-0004-Documentation-for-FOR-ALL-TABLES-IN-SCHEMA-publi.patch text/x-patch 15.2 KB
v35-0005-Implemented-pg_publication_objects-view.patch text/x-patch 6.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro Horiguchi 2021-09-30 10:11:16 Re: Logical replication keepalive flood
Previous Message Amit Kapila 2021-09-30 10:07:47 Re: pgsql: Document XLOG_INCLUDE_XID a little better