Re: Data is copied twice when specifying both child and parent table in publication

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Greg Nancarrow <gregn4422(at)gmail(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, Amit Langote <amitlangote09(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: Re: Data is copied twice when specifying both child and parent table in publication
Date: 2021-11-12 04:11:01
Message-ID: CAA4eK1+NWreG=2sKiMz8vFzTsFhEHCjgQMyAu6zj3sdLmcheYg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Nov 11, 2021 at 1:44 PM Greg Nancarrow <gregn4422(at)gmail(dot)com> wrote:
>
> On Thu, Nov 11, 2021 at 5:52 PM houzj(dot)fnst(at)fujitsu(dot)com
> <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> >
> > When looking into how to fix the second issue, I have a question:
> >
> > After changing publish_via_partition_root from false to true, the
> > subcriber will fetch the partitioned table from publisher when refreshing.
> >
> > In subsriber side, If all the child tables of the partitioned table already
> > subscribed, then we can just skip the table sync for the partitioned table. But
> > if only some of the child tables(not all child tables) were already subscribed,
> > should we skip the partitioned table's table sync ? I am not sure about the
> > appropriate behavior here.
> >
> > What do you think ?
> >
>
> I'm not sure you can skip the partitioned table's table sync as you
> are suggesting, because on the subscriber side, the tables are mapped
> by name, so what is a partitioned table on the publisher side might
> not be a partitioned table on the subscriber side (e.g. might be an
> ordinary table; and similarly for the partitions) or it might be
> partitioned differently to that on the publisher side.
>

Sure, we don't know about that, or at least there is no such mapping
that is recorded. So, I think we should skip it even if any one of the
child table is present.

> (I might be
> wrong here, and I don't have a good solution, but I can see the
> potential for inconsistent data resulting in this case, unless say,
> the subscriber "child tables" are first truncated on the refresh, if
> they are in fact partitions of the root, and then the table sync
> publishes the existing data via the root)
>

Do you want to say current behavior for this case where data is copied
twice is okay? I think we need to find a way to handle this and
document it to set the expectations right.

--
With Regards,
Amit Kapila.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2021-11-12 04:27:39 Re: Data is copied twice when specifying both child and parent table in publication
Previous Message vignesh C 2021-11-12 04:08:35 Re: Optionally automatically disable logical replication subscriptions on error