Re: Bug in UTF8-Validation Code?

> Andrew - Supernews <andrew+nonews(at)supernews(dot)com> writes:
> > Thinking about this made me realize that there's another, ahem, elephant
> > in the room here: convert().
> > By definition convert() returns text strings which are not valid in the
> > server encoding. How can this be addressed?
>
> Remove convert(). Or at least redefine it as dealing in bytea not text.

That would break some important use cases.

1) A user have UTF-8 database which contains various language
data. Each language has its own table. He wants to sort a SELECT
result by using ORDER BY. Since locale cannot handle multiple
languages, he uses C locale and do the SELECT something like this:

SELECT * FROM french_table ORDER BY convert(t, 'LATIN1');
SELECT * FROM japanese_table ORDER BY convert(t, 'EUC_JP');

2) A user has a UTF-8 database but unfortunately his OS's UTF-8 locale
is broken. He decided to use C locale and want to sort the result
from SELECT like this.

SELECT * FROM japanese_table ORDER BY convert(t, 'EUC_JP');

Note that sorting by UTF-8 physical order would produce random
results. So following would not help him in this case:

SELECT * FROM japanese_table ORDER BY t;

Also I don't understand what this is different to the problem when we
have a message catalogue which does not match the encoding.
--
Tatsuo Ishii
SRA OSS, Inc. Japan