Re: invalid byte sequence for encoding "UTF8": 0xf481 - how could this happen?

From: "Albe Laurenz" <laurenz(dot)albe(at)wien(dot)gv(dot)at>
To: "Rural Hunter *EXTERN*" <ruralhunter(at)gmail(dot)com>, <pgsql-admin(at)postgresql(dot)org>
Subject: Re: invalid byte sequence for encoding "UTF8": 0xf481 - how could this happen?
Date: 2012-04-16 08:31:09
Message-ID: D960CB61B694CF459DCFB4B0128514C207BB6CE6@exadv11.host.magwien.gv.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Rural Hunter wrote:
> My db is in utf-8, I have a row in my table say tmp_article and I
wanted
> to generate ts_vector from the article content:
> select to_tsvector(content) from tmp_article;
> But I got this error:
> ERROR: invalid byte sequence for encoding "UTF8": 0xf481
>
> I am wondering how this could happen. I think if there was invalid
UTF8
> bytes in the content, it shouldn't have been able to inserted into the
> tmp_article table as I sometimes see similar errors when inserting
> records to tmp_article. Am I right?

You are right in theory. A lot depends on your PostgreSQL version,
because
the efforts to prevent invalid strings from entering the database have
led to changes over the versions. Older versions are more permissive.

To test the theory that the contents of the table are bad, you can
test if the same happens if you

SELECT convert_to(content, 'UTF8') FROM tmp_article;

Yours,
Laurenz Albe

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Chris Ernst 2012-04-16 08:32:28 Re: Recreate primary key without dropping foreign keys?
Previous Message Uwe Bartels 2012-04-16 08:17:10 enterprisedb package and pam on debian