Re: Allow logical replication to copy tables in binary format

shiy(dot)fnst(at)fujitsu(dot)com <shiy(dot)fnst(at)fujitsu(dot)com>, 23 Şub 2023 Per, 12:29
tarihinde şunu yazdı:

> On Thu, Feb 23, 2023 12:40 PM Kuroda, Hayato/黒田 隼人 <
> kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
> >
> > > > I'm not sure the combination of "copy_format = binary" and
> "copy_data =
> > false"
> > > > should be accepted or not. How do you think?
> > >
> > > It seems quite useless indeed to specify the format of a copy that
> won't
> > happen.
> >
> > I understood that the conbination of "copy_format = binary" and
> "copy_data =
> > false"
> > should be rejected in parse_subscription_options() and
> AlterSubscription(). Is it
> > right?
> > I'm expecting that is done in next version.
> >
>
> The copy_data option only takes effect once in CREATE SUBSCIPTION or ALTER
> SUBSCIPTION REFRESH PUBLICATION command, but the copy_format option can
> take
> affect multiple times if the subscription is refreshed multiple times.
> Even if
> the subscription is created with copy_date=false, copy_format can take
> affect
> when executing ALTER SUBSCIPTION REFRESH PUBLICATION. So, I am not sure we
> want
> to reject this usage.
>

I believe the places copy_data and copy_format are needed are pretty much
the same. I couldn't think of a case where copy_format is needed but
copy_data isn't. Please let me know if I'm missing something.
CREATE SUBSCRIPTION, ALTER SUBSCRIPTION SET/ADD/REFRESH PUBLICATION are all
the places where initial sync can happen. For all these commands, copy_data
needs to be given as a parameter or it will be set to the default value
which is true. Even if copy_data=false when the sub was created, REFRESH
PUBLICATION (without an explicit copy_data parameter) will copy some tables
if needed regardless of what copy_data was in CREATE SUBSCRIPTION. This is
because copy_data is not something stored in pg_subscription or another
catalog. But this is not an issue for copy_fornat since its value will be
stored in the catalog. This can allow users to set the format even if
copy_data=false and no initial sync is needed at that moment. So that
future initial syncs which can be triggered by ALTER SUBSCRIPTION will be
performed in the correct format.
So, I also think we should allow setting copy_format even if
copy_data=false.

Another way to deal with this issue could be expecting the user to specify
format every time copy_format is needed, similar to the case for copy_data,
and moving on with text (default) format if it's not specified for the
current CREATE/ALTER SUBSCRIPTION execution. But I don't think this would
make things easier.

Best,
--
Melih Mutlu
Microsoft