Re: Allow logical replication to copy tables in binary format

From: Jelte Fennema <postgres(at)jeltef(dot)nl>
To: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
Cc: Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>, "Takamichi Osumi (Fujitsu)" <osumi(dot)takamichi(at)fujitsu(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, Euler Taveira <euler(at)eulerto(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
Subject: Re: Allow logical replication to copy tables in binary format
Date: 2023-02-22 10:43:23
Message-ID: CAGECzQSStdb+x1BxzCktZd1uSjvd6eYq1wcHV3vPCykrGqxYKQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> If in future version the general data type is added, the copy command in binary
> format will not work well, right? It is because the inferior version does not have
> recv/send functions for added type. It means that there is a possibility that
> replication between different versions may be failed if binary format is specified.
> Therefore, I think we should accept copy_format = binary only when the major
> version of servers are the same.

I don't think it's necessary to check versions. Yes, there are
situations where binary will fail across major versions. But in many
cases it does not. To me it seems the responsibility of the operator
to evaluate this risk. And if the operator chooses wrong and uses
binary copy across incompatible versions, then it will still fail hard
in that case during the copy phase (so still a very early error). So I
don't see a reason to check pre-emptively, afaict it will only
disallow some valid usecases and introduce more code.

Furthermore no major version check is done for "binary = true" either
(afaik). The only additional failure scenario that copy_format=binary
introduces is when one of the types does not implement a send function
on the source. With binary=true, this would continue to work, but with
copy_format=binary this stops working. All other failure scenarios
that binary encoding of types introduces apply to both binary=true and
copy_format=binary (the only difference being in which phase of the
replication these failures happen, the apply or the copy phase).

> I'm not sure the combination of "copy_format = binary" and "copy_data = false"
> should be accepted or not. How do you think?

It seems quite useless indeed to specify the format of a copy that won't happen.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2023-02-22 10:48:15 LWLock deadlock in brinRevmapDesummarizeRange
Previous Message Peter Eisentraut 2023-02-22 10:32:19 Re: Transparent column encryption