Re: Support logical replication of DDLs

From: shveta malik <shveta(dot)malik(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Michael Paquier <michael(at)paquier(dot)xyz>, "Wei Wang (Fujitsu)" <wangw(dot)fnst(at)fujitsu(dot)com>, "Yu Shi (Fujitsu)" <shiy(dot)fnst(at)fujitsu(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, Ajin Cherian <itsajin(at)gmail(dot)com>, Runqi Tian <runqidev(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, li jie <ggysxcq(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, rajesh singarapu <rajesh(dot)rs0541(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Zheng Li <zhengli10(at)gmail(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: Re: Support logical replication of DDLs
Date: 2023-06-30 07:06:07
Message-ID: CAJpy0uAhLjQZ0Dh0KWDFP8mrnG0rbx99_heavwn8Ke8ZuD-Umg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Thu, Jun 22, 2023 at 9:39 AM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
>
> On Mon, Jun 19, 2023 at 3:04 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> >
> > In insert_table_elements(), sometimes we access system twice for each
> > of the columns and this is to identify the above case where no
> > elements are present. Would it be better if simply for zero element
> > object array in this case and detect the same on the receiving side?
> > If this is feasible then we can simply name the function as
> > add_table_elems/add_table_elements. Also, in this context, can we
> > change the variable name telems to telems_present to make it bit easy
> > to follow.
>
> Modified telems to telems_present. I am reviewing the first part.
> Please allow some more time.
>

Fixed this. Now we always add table_elements array. If there are no
elements then it will be empty []. Regarding '()' around
table-elements, we take a call once we are done adding table-elements.
We peek into jsonb parse-state to figure out if we have actually added
anything and then decide what goes to "fmt" string. Different
scenarios are mentioned in code-comments in add_table_elems().

Apart from this, addressed below:

1) Fixed a bug where tables with serial columns (sequences) were not
replicated. Thanks Ajin for providing the fix.

2) Fixed a few more things:
a) Removed 'toJsonb' from all function names. Earlier it was this way
to distinguish from object-tree functions since we were maintaining
both the patches.
b) Renamed and moved a few helper functions which were not actually
for deparsing purposes to relevant files.
c) Removed role_to_jsonb_element(), 'myowner' can now be added like
any other key-value pair using new_jsonb_VA().
d) Optimized and refactored code at many places.
e) Improved/corrected comments.

The given patches do not include Vignesh's auto-generation work in
[1], but if there is interest from reviewers, we can rebase those as
well.

[1]: https://www.postgresql.org/message-id/CALDaNm0Rvv2EWOOQeGc_1j3b3ENceDRO8jd%2Bgbf0Y_J7d1FGGw%40mail.gmail.com

thanks
Shveta

Attachment Content-Type Size
0003-Enhance-the-event-trigger-to-support-DDL--2023_06_30.patch application/octet-stream 13.5 KB
0002-Deparser-for-Alter-Table-DDL-commands-2023_06_30.patch application/octet-stream 58.6 KB
0001-Deparser-for-Create-And-Drop-Table-DDL-co-2023_06_30.patch application/octet-stream 98.1 KB
0005-Apply-the-DDL-change-as-that-same-user-th-2023_06_30.patch application/octet-stream 58.8 KB
0004-DDL-replication-for-Table-DDL-commands-2023_06_30.patch application/octet-stream 238.9 KB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Ashok Patil 2023-06-30 08:17:29 Re: Query regarding managing Replication
Previous Message David G. Johnston 2023-06-30 03:27:59 Re: psql and pgpass.conf on Windows

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2023-06-30 07:09:03 Re: [PATCH] Honor PG_TEST_NOCLEAN for tempdirs
Previous Message Amit Langote 2023-06-30 06:54:38 Re: ExecRTCheckPerms() and many prunable partitions (checkAsUser)