Re: Support logical replication of DDLs

From: shveta malik <shveta(dot)malik(at)gmail(dot)com>
To: "Yu Shi (Fujitsu)" <shiy(dot)fnst(at)fujitsu(dot)com>
Cc: vignesh C <vignesh21(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, Ajin Cherian <itsajin(at)gmail(dot)com>, "Wei Wang (Fujitsu)" <wangw(dot)fnst(at)fujitsu(dot)com>, Runqi Tian <runqidev(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, li jie <ggysxcq(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, rajesh singarapu <rajesh(dot)rs0541(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Zheng Li <zhengli10(at)gmail(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: Re: Support logical replication of DDLs
Date: 2023-06-05 09:30:01
Message-ID: CAJpy0uBwCZCniPR6vh26L+wpSf4xzUN8omUa9DzF-x1CAud_xA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Thu, Jun 1, 2023 at 1:10 PM Yu Shi (Fujitsu) <shiy(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Wed, May 31, 2023 5:41 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
> >
> > On Mon, May 29, 2023 at 11:45 AM Yu Shi (Fujitsu) <shiy(dot)fnst(at)fujitsu(dot)com>
> > wrote:
> > >
> > > 0008 patch
> > > -----
> > > 4.
> > > case AT_AddColumn:
> > > /* XXX need to set the "recurse" bit somewhere? */
> > > Assert(IsA(subcmd->def, ColumnDef));
> > > - tree = deparse_ColumnDef(rel, dpcontext, false,
> > > - (ColumnDef *) subcmd->def, true,
> > &expr);
> > >
> > > mark_function_volatile(context, expr);
> > >
> > > After this change, `expr` is not assigned a value when mark_function_volatile is
> > called.
> > >
> >
> > Corrected.
> >
>
> It looks the call to mark_function_volatile() is removed in this case. I think
> we still need it, otherwise we won't know if the command contains a volatile
> function. (see check_command_publishable().)
>

Modified this. Thanks for pointing this out.

> > > 5.
> > > create table p1(f1 int);
> > > create table p1_c1() inherits(p1);
> > > alter table p1 add constraint inh_check_constraint1 check (f1 > 0);
> > > alter table p1_c1 add constraint inh_check_constraint1 check (f1 > 0);
> > >
> > > The re-formed command of the last command is "ALTER TABLE public.p1_c1",
> > which
> > > seems to be wrong.
> > >
> >
> > Fixed, second alter-table should actually be no-op in terms of
> > deparsing. But when it is run without running the first alter-table
> > command, it should generate the reformed command.
> >
>
> If the second alter-table is no-op in terms of deparsing, the dumped result of
> origin command and that of re-formed command will be different. This seems to be
> because `conislocal` column of pg_constraint has different values. (After the
> second alter-table, its value is changed from false to true.)
>

Okay, I see. I am analyzing it to figure out if we really need the
second call to be replicated on sub. It needs more review around the
impact of 'conislocal'. I will respond once concluded.

Please find the new set of patches attached. The changes are in 0008
only. New changes are majorly for:
--comments given by Shi-san. Thanks Vignesh for working on this.
--addition of more test-cases to cover missing scenarios. Thanks
Shi-san for working on this.

thanks
Shveta

Attachment Content-Type Size
0003-Add-verbose-option-for-ddl-deparse-module-2023_06_05.patch application/octet-stream 47.1 KB
0001-Deparser-for-Table-DDL-commands-and-exten-2023_06_05.patch application/octet-stream 167.3 KB
0002-Enhance-the-event-trigger-to-support-DDL--2023_06_05.patch application/octet-stream 33.9 KB
0005-DDL-replication-for-Table-DDL-commands-2023_06_05.patch application/octet-stream 248.2 KB
0004-Introduce-the-test_ddl_deparse_regress-te-2023_06_05.patch application/octet-stream 977.8 KB
0006-Add-subscription-tap-test-for-DDL-replica-2023_06_05.patch application/octet-stream 20.8 KB
0007-Apply-the-DDL-change-as-that-same-user-th-2023_06_05.patch application/octet-stream 60.0 KB
0008-ObjTree-Removal-for-multiple-commands-2023_06_05.patch application/octet-stream 1.8 MB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message gzh 2023-06-05 09:38:55 Re:Re: Is there any good optimization solution to improve the query efficiency?
Previous Message David Rowley 2023-06-05 08:21:19 Re: Is there any good optimization solution to improve the query efficiency?

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2023-06-05 10:31:41 Re: [PoC] Improve dead tuple storage for lazy vacuum
Previous Message Joel Jacobson 2023-06-05 09:27:53 Re: Do we want a hashset type?