Re: Support logical replication of DDLs

From: vignesh C <vignesh21(at)gmail(dot)com>
To: Ajin Cherian <itsajin(at)gmail(dot)com>
Cc: Zheng Li <zhengli10(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, li jie <ggysxcq(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, rajesh singarapu <rajesh(dot)rs0541(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Support logical replication of DDLs
Date: 2023-01-19 08:51:01
Message-ID: CALDaNm0dOwbPVrkqw9OLN=TVFdo5aroUNwdjG5SAcKQg93-g0w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Mon, 16 Jan 2023 at 19:04, Ajin Cherian <itsajin(at)gmail(dot)com> wrote:
>
> On Mon, Jan 16, 2023 at 2:13 PM vignesh C <vignesh21(at)gmail(dot)com> wrote:
> >
> > On Sun, 15 Jan 2023 at 09:39, Ajin Cherian <itsajin(at)gmail(dot)com> wrote:
> > >
> > > On Fri, Jan 13, 2023 at 5:33 PM vignesh C <vignesh21(at)gmail(dot)com> wrote:
> > > > Adding support for CREATE/ALTER/DROP Publication ddl deparsing.
> > > > The attached v61 patch has the changes for the same.
> > > >
> > >
> > > Hi Vignesh,
> > > this doesn't seem to compile:
> > >
> > > gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith
> > > -Wdeclaration-after-statement -Werror=vla -Wendif-labels
> > > -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing
> > > -fwrapv -fexcess-precision=standard -g -ggdb -Og -g3
> > > -fno-omit-frame-pointer -I../../../src/include -D_GNU_SOURCE -c -o
> > > ddl_deparse.o ddl_deparse.c
> > > ddl_deparse.c: In function ‘deparse_PublicationObjects’:
> > > ddl_deparse.c:8956:3: error: unknown type name ‘publication_rel’
> > > publication_rel *pub_rel = (publication_rel *) lfirst(lc1);
> > > ^
> > > ddl_deparse.c:8956:31: error: ‘publication_rel’ undeclared (first use
> > > in this function)
> > > publication_rel *pub_rel = (publication_rel *) lfirst(lc1);
> > > ^
> > > ddl_deparse.c:8956:31: note: each undeclared identifier is reported
> > > only once for each function it appears in
> > > ddl_deparse.c:8956:48: error: expected expression before ‘)’ token
> > > publication_rel *pub_rel = (publication_rel *) lfirst(lc1);
> >
> > I was able to apply all the patches and compile successfully, also
> > cfbot as in [1] has compiled successfully. One thing I noticed is that
> > structure is available at 006 patch, if you did not apply all the
> > patch it might fail, we will take care of structure movement in the
> > next version.
> > [1] - https://cirrus-ci.com/task/5231672585617408
> >
>
> Fixing an earlier comment from Peter:
> On Mon, Oct 31, 2022 at 7:07 PM Peter Smith
> <smithpb2250(at)gmail(dot)com> wrote:
> >
> >0. append_object_to_format_string
> >Instead of this little loop why doesn't the code just look for the
> ?name delimiters?
> >
> >e.g.
> >pstart = strch(sub_fmt, '{');
> >pend = strbrk(pstart, ":}");
> >
> >then the 'name' is what lies in between...
>
> made the logic simpler with strchr.
>
> Also:
> 1. fixed a compilation warning seen in publicationcmds.c
> 2. fixed a broken documentation build
> 3. changed a failed build of patch 3 due to missing header.

Hi,

I have fixed the following issues in the v63 version patch attached:
1) pg_dump was dumping the internally generated event triggers and the
create publication with ddl option which resulted in creating multiple
event triggers while restoring, this patch now addresses this by
skipping the dump of internally generated event triggers
2) Restoring of non ddl publication was not correct, this is handled
3) There were few indent issues while applying 003 patch.

These are handled in the v63 version attached.

Regards,
Vignesh

Attachment Content-Type Size
v63-0001-Infrastructure-to-support-DDL-deparsing.patch text/x-patch 37.5 KB
v63-0005-DDL-messaging-infrastructure-for-DDL-replication.patch text/x-patch 41.5 KB
v63-0004-Introduce-the-test_ddl_deparse_regress-test-modu.patch text/x-patch 47.4 KB
v63-0003-Support-DDL-deparse-of-the-rest-commands.patch text/x-patch 198.7 KB
v63-0002-Functions-to-deparse-Table-DDL-commands.patch text/x-patch 131.0 KB
v63-0007-Document-DDL-replication-and-DDL-deparser.patch text/x-patch 40.6 KB
v63-0006-Support-DDL-replication.patch text/x-patch 210.2 KB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Harmen 2023-01-19 09:04:25 Re: row estimate for partial index
Previous Message Amit Kapila 2023-01-19 07:05:38 Re: Support logical replication of DDLs

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2023-01-19 08:52:57 Re: [PATCH] Constify proclist.h
Previous Message Michael Paquier 2023-01-19 08:46:37 Re: Generating code for query jumbling through gen_node_support.pl