Re: Logical Replication and Character encoding

On 2 February 2017 at 11:45, Euler Taveira <euler(at)timbira(dot)com(dot)br> wrote:

> I don't think storage without conversion is an acceptable approach. We
> should provide options to users such as ignore tuple or NULL for
> column(s) with conversion problem. I wouldn't consider storage data
> without conversion because it silently show incorrect data and we
> historically aren't flexible with conversion routines.

pglogical and BDR both require identical encoding; they test for this
during startup and refuse to replicate if the encoding differs.

For the first pass at core I suggest a variant on that policy: require
source and destination encoding to be the same. This should probably
be the first change, since it definitively prevents the issue from
arising.

If time permits we could also allow destination encoding to be UTF-8
(including codepage 65001) with any source encoding. This requires
encoding conversion to be performed, of course.

The downside is that this will impact users who use a common subset of
two encodings. This is most common for Windows-1252 <-> ISO-8859-15
(or -1 if you're old-school) but also arises anywhere the common 7 bit
subset is used. Until we can define an encoding exception policy
though, I think we should defer supporting those and make them a
"later" problem.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services