Re: Support logical replication of DDLs

From: Zheng Li <zhengli10(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, li jie <ggysxcq(at)gmail(dot)com>, Ajin Cherian <itsajin(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, rajesh singarapu <rajesh(dot)rs0541(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Support logical replication of DDLs
Date: 2023-01-24 03:27:50
Message-ID: CAAD30UJaP-eEChY-i0uEOn3sNZpqSD7Ma-abDy4s+JvxG=QdHw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

> > Yes, CREATE/ALTER SERVER commands are also supported by the current
> > DDL replication patch.
> >
> > >But what about data inserted by the publisher on the
> > > foreign server?
> >
> > I thought the data inserted to a foreign table will always be stored
> > on the foreign server unless I'm mistaken?
> >
>
> I also have the same understanding. It is not clear to me if there is
> a use case to just allow the foreign server set up without caring for
> data replication. So, what this will achieve is both publisher and
> subscriber will be allowed to perform operations on the same foreign
> server but not sure if that is expected by the user and is useful to
> them.
One use case I think of is using a logical replica for online major
version upgrade, users would want the foreign server on the
subscriber/replica set up identical to the publisher.
There may be cases when foreign server replication is not needed,
which justifies the motivation to define fine grained DDL replication
levels.

>
> > > > > We should also think
> > > > > about initial sync for all those objects as well.
> > > >
> > > > Agree, we're starting an investigation on initial sync. But I think
> > > > initial sync depends on
> > > > DDL replication to work reliably, not the other way around. DDL replication can
> > > > work on its own without the initial sync of schema, users just need to
> > > > setup the initial
> > > > schema just like they would today.
> > > >
> > >
> > > The difference is that today users need to take care of all schema
> > > setup on both and follow changes in the same on the publisher. But
> > > with DDL replication, there has to be a point prior to which both the
> > > nodes have the same setup. For that, before setting up DDL
> > > replication, users need to ensure that both nodes have the same
> > > schema, and then during setup, the user doesn't perform any DDL on the
> > > publisher.
> >
> > The users can perform DDL during the setup if they do the following:
> > 1. Create a logical replication slot to capture changes on the publisher
> > 2. Do a backup for the publisher
> > 3. Restore the backup as the subscriber
> > 4. Advance the logical slot to the last valid LSN of the restore
> > 5. Create pub/sub and use the above logical slot.
> >
>
> Are you talking about basebackup/restore or pg_dump? I have later in
> mind (with the snapshot option) to achieve it. However, I think it
> won't be convenient for users to do those steps by themselves as there
> is a risk of mistakes leading to wrong set up.

I'm talking about basebackup where the last valid LSN can be
identified from the Postgres logs. I don't think we can easily
identify the LSN to advance the slot to with pg_dump/pg_restore since
it's a logical copy.
I agree these steps are not straightforward, but some customers are
doing it themselves.

Regards,
Zane

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Zwettler Markus (OIZ) 2023-01-24 15:37:41 Is a logical replication crash recovery safe?
Previous Message Amit Kapila 2023-01-23 08:46:52 Re: Support logical replication of DDLs

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2023-01-24 03:28:06 Re: postgres_fdw uninterruptible during connection establishment / ProcSignalBarrier
Previous Message Andres Freund 2023-01-24 03:23:27 Test failures of 100_bugs.pl