Re: Postgres Encoding conversion problem

On 04/22/2008 05:37 PM, Albe Laurenz wrote:
> Clemens Schwaighofer wrote:
>> I sometimes have a problem with conversion of encodings eg from UTF-8
>> tio ShiftJIS:
>>
>> ERROR: character 0xf0a0aeb7 of encoding "UTF8" has no
>> equivalent in "SJIS"
>>
>> I have no idea what character this is, I cannot view it in my
>> browser, etc.
>
> It translates to Unicode 10BB7, which is not defined.
> I guess that is not intended; can you guess what the character(s) should be?

to be honest no idea. its some chinese character, I have no idea how the
user input this, because this is a japanese page.

I actually found the carachter, but only my Mac OS X can show it. It
looks similar to a japanese character used for a name, but how the
chinese one got selected is a mystery to me ...

>> If I run the conversion through PHP with mb_convert_encoding it works,
>> perhaps he is ignoring the character.
>>
>> Is there a way to do a similar thing, like ignoring this character in
>> postgres too?
>
> As far as I know, no.
> You'll have to fix the data before you import them.

well, the web page & data is in utf8 so I never see this issue, except I
would write a method that detects illegal shift_jis characters, and
thats difficult.

The reporting is only done in CSV ... so I am not sure if it is worth to
waste too much time here.

thanks for the tip.

--
[ Clemens Schwaighofer -----=====:::::~ ]
[ IT Engineer/Manager, TEQUILA\ Japan IT Group ]
[ 6-17-2 Ginza Chuo-ku, Tokyo 104-8167, JAPAN ]
[ Tel: +81-(0)3-3545-7703 Fax: +81-(0)3-3545-7343 ]
[ http://www.tequila.co.jp ]