Re: Logical Replication and Character encoding

On 4/5/17 21:32, Kyotaro HORIGUCHI wrote:
> At Wed, 5 Apr 2017 11:33:51 -0400, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote in <5401fef6-c0c0-7e8a-d8b1-169e30cbd854(at)2ndquadrant(dot)com>
>> After further thinking, I prefer the alternative approach of using
>> pq_sendcountedtext() as is and sticking the trailing zero byte on on the
>> receiving side. This is a more localized change, and keeps the logical
>> replication protocol consistent with the main FE/BE protocol. (Also, we
>> don't need to send a useless byte around.)
>
> I'm not sure about the significance of the trailing zero in the
> the logical replication protocol.

It doesn't matter. It could be "nice" to have it because then the
receiving side doesn't need to add it if it wants to assemble a C
string. But for the reasons above, it doesn't seem worth doing that.

> walreceiver stops on the opposite side with the following
> complaint.
>
>> ERROR: could not receive data from WAL stream: ERROR: character with byte sequence 0xe6 0xbc 0xa2 in encoding "UTF8" has no equivalent in encoding "LATIN1"
>> CONTEXT: COPY t, line 1: ""
>> LOG: worker process: logical replication worker for subscription 16391 sync 16384 (PID 26915) exited with exit code 1
>
> After this, walreceiver repeats reconnecting to master with no
> wait. Maybe walreceiver had better refrain from reconnection
> after certain kinds of faiure but it is not an urgent issue.

This is controlled by wal_retrieve_retry_interval.

I see what you are saying about treating certain errors as potentially
permanent, but I think a lot more work would need to be done to be able
to manage that.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services