Re: possible encoding issues with libxml2 functions

On Mon, Feb 20, 2017 at 07:48:18PM +0100, Pavel Stehule wrote:
> Today I played with xml_recv function and with xml processing functions.
>
> xml_recv function ensures correct encoding from document encoding to server
> encoding. But the decl section holds original encoding info - that should
> be obsolete after encoding. Sometimes we solve this issue by removing decl
> section - see the xml_out function.
>
> Sometimes we don't do it - lot of functions uses direct conversion from
> xmltype to xmlChar.

> There are possible two fixes
>
> a) clean decl on input - the encoding info can be removed from decl part
>
> b) use xml_out_internal everywhere before transformation to
> xmlChar. pg_xmlCharStrndup can be good candidate.

I'd prefer (a) if the xml type were a new feature, because no good can come of
storing an encoding in each xml field when we know the actual encoding is the
database encoding. However, if you implemented (a), we'd still see untreated
values brought over via pg_upgrade. Therefore, I would try (b) first. I
suspect the intent of xml_parse() was to implement (b); it will be interesting
to see your test case that malfunctions.