Re: UTF16 surrogate pairs in UTF8 encoding

On 9/8/10, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Marko Kreen <markokr(at)gmail(dot)com> writes:
> > Although it does seem unnecessary.
>
>
> The reason I asked for this to be spelled out is that ordinarily,
> a backslash escape \nnn is a very low-level thing that will insert
> exactly what you say. To me it's quite unexpected that the system
> would editorialize on that to the extent of replacing two UTF16
> surrogate characters by a single code point. That's necessary for
> correctness because our underlying storage is UTF8, but it's not
> obvious that it will happen. (As a counterexample, if our underlying
> storage were UTF16, then very different things would need to happen
> for the exact same SQL input.)
>
> I think a lot of people will have this same question when reading
> this para, which is why I asked for an explanation there.

Ok, but I still don't like the "when"s. How about:

- 6-digit form technically makes this unnecessary. (When surrogate
- pairs are used when the server encoding is <literal>UTF8</>, they
- are first combined into a single code point that is then encoded
- in UTF-8.)
+ 6-digit form technically makes this unnecessary. (Surrogate
+ pairs are not stored directly, but combined into a single
+ code point that is then encoded in UTF-8.)

--
marko