Re: multibyte-character aware support for function "downcase_truncate_identifier()"

On Sun, Nov 21, 2010 at 6:48 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> On 11/21/2010 06:09 PM, Robert Haas wrote:
>>> I think that's fair.  It actually doesn't seem like it should be that
>>> hard if we knew that the server encoding were UTF8 - it's just a big
>>> translation table somewhere, no?
>
>> No, it's far more complex. See for example
>> <http://unicode.org/reports/tr21/tr21-3.html>, which says:
>
> Yeah.  I'm actually not sure that the SQL committee has thought very
> hard about this, because the spec is worded as though they think that
> "Unicode case normalization" is all they have to say to uniquely define
> what to do.  The Unicode guys recognize that case mapping is
> locale-specific, which puts us right back at square one.  But leaving
> spec compliance aside, we know from bitter experience that we cannot use
> a definition that lets the Turkish locale fool with the mapping of i/I.
> I suspect that locale-dependent mappings of any other characters are
> just as bad, we simply haven't had enough users burnt by such cases to
> have an institutional memory of it.  But for example do you really think
> it's a good idea if pg_dump and reload into a DB with a different locale
> results in changing the normalized form of SQL identifiers?

No, especially if it results in queries that used to work breaking,
which it well could. But I'm not sure where to go with it from there,
beyond throwing up my hands.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company