Skip site navigation (1) Skip section navigation (2)

Re: Postgres Encoding conversion problem

From: Clemens Schwaighofer <cs(at)tequila(dot)co(dot)jp>
To: Albe Laurenz <laurenz(dot)albe(at)wien(dot)gv(dot)at>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Postgres Encoding conversion problem
Date: 2008-04-22 08:43:23
Message-ID: 480DA52B.20702@tequila.co.jp (view raw or flat)
Thread:
Lists: pgsql-general
On 04/22/2008 05:37 PM, Albe Laurenz wrote:
> Clemens Schwaighofer wrote:
>> I sometimes have a problem with conversion of encodings eg from UTF-8
>> tio ShiftJIS:
>>
>> ERROR:  character 0xf0a0aeb7 of encoding "UTF8" has no
>> equivalent in "SJIS"
>>
>> I have no idea what character this is, I cannot view it in my
>> browser, etc.
> 
> It translates to Unicode 10BB7, which is not defined.
> I guess that is not intended; can you guess what the character(s) should be?

to be honest no idea. its some chinese character, I have no idea how the
user input this, because this is a japanese page.

I actually found the carachter, but only my Mac OS X can show it. It
looks similar to a japanese character used for a name, but how the
chinese one got selected is a mystery to me ...

>> If I run the conversion through PHP with mb_convert_encoding it works,
>> perhaps he is ignoring the character.
>>
>> Is there a way to do a similar thing, like ignoring this character in
>> postgres too?
> 
> As far as I know, no.
> You'll have to fix the data before you import them.

well, the web page & data is in utf8 so I never see this issue, except I
would write a method that detects illegal shift_jis characters, and
thats difficult.

The reporting is only done in CSV ... so I am not sure if it is worth to
waste too much time here.

thanks for the tip.

-- 
[ Clemens Schwaighofer                      -----=====:::::~ ]
[ IT Engineer/Manager, TEQUILA\ Japan IT Group               ]
[                6-17-2 Ginza Chuo-ku, Tokyo 104-8167, JAPAN ]
[ Tel: +81-(0)3-3545-7703            Fax: +81-(0)3-3545-7343 ]
[ http://www.tequila.co.jp                                   ]

In response to

Responses

pgsql-general by date

Next:From: wstrzalkaDate: 2008-04-22 09:41:50
Subject: plpgsql and logical expression evaluation
Previous:From: Albe LaurenzDate: 2008-04-22 08:37:59
Subject: Re: Postgres Encoding conversion problem

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group