Re: Support logical replication of DDLs

From: Zheng Li <zhengli10(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Euler Taveira <euler(at)eulerto(dot)com>, japin <japinli(at)hotmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, rajesh(dot)rs0541(at)gmail(dot)com, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Support logical replication of DDLs
Date: 2022-04-13 17:22:46
Message-ID: CAAD30UKJJXSMCb0Xk9ZtS1si4KzbasmYzbJjD8rZxTX2HWnGZA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

>Thanks, but it would be good if you can share it in the patch form as
>well unless you need to send patches too frequently. It is easier to
>give comments.

The rebased patches are attached. They apply on commit ed0fbc8.

> > I agree with the point that their design can not be completely
> > independent. They have some logical relationship of what schema will
> > be copied by the initial sync and where is the exact boundary from
> > which we will start sending as replication. And suppose first we only
> > plan to implement the replication part then how the user will know
> > what all schema user has to create and what will be replicated using
> > DDL replication? Suppose the user takes a dump and copies all the
> > schema and then creates the subscription, then how we are we going to
> > handle the DDL concurrent to the subscription command?
> >
>
> Right, I also don't see how it can be done in the current
> implementation. So, I think even if we want to develop these two as
> separate patches they need to be integrated to make the solution
> complete.

The concurrent DDL issue actually exists today even with DML
replication. i.e. user have to make sure table schema is in sync
before starting logical replication.

Here is a high level idea to implement schema initialization:
1. On source DB: dump the schema and create a logical replication slot
on a snapshot at the same time
2. On target DB: restore the schema
3. On target DB: create subscription by pointing it to the logical
replication slot and the snapshot in step 1, so that the initial sync
is based on the snapshot and concurrent changes are replicated using
the logical replication slot.

Regards,
Zheng

Attachment Content-Type Size
v4-0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch application/octet-stream 8.0 KB
v4-0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch application/octet-stream 65.0 KB
v4-0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch application/octet-stream 4.1 KB
v4-0003-Logical-replication-of-DDL-messages.patch application/octet-stream 41.2 KB
v4-0002-Support-logical-logging-and-decoding-of-DDL-command-.patch application/octet-stream 61.3 KB
v4-0006-Add-couple-tests-for-DDL-replication-case.patch application/octet-stream 4.5 KB

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Perryn Fowler 2022-04-14 07:10:55 Is this a reasonable use for advisory locks?
Previous Message Amit Kapila 2022-04-13 09:50:20 Re: Support logical replication of DDLs

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2022-04-13 17:29:34 Re: typos
Previous Message chap 2022-04-13 16:48:10 Re: timezones BCE