Skip site navigation (1) Skip section navigation (2)

Re: UTF-8 and LIKE vs =

From: David Wheeler <david(at)kineticode(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Ian Barwick <barwick(at)gmail(dot)com>, pgsql-general(at)postgresql(dot)org
Subject: Re: UTF-8 and LIKE vs =
Date: 2004-08-23 22:50:07
Message-ID: C83F7824-F556-11D8-990D-000A95972D84@kineticode.com (view raw or flat)
Thread:
Lists: pgsql-general
On Aug 23, 2004, at 3:44 PM, Tom Lane wrote:

>> Yes, it means that = is doing the wrong thing!!
>
> I have seen this happen in situations where the strings contained
> character sequences that were illegal according to the encoding that 
> the
> locale thought was in force.  (It seems that strcoll() will return more
> or less random results in such cases...)  In particular, given that you
> have
>
>> LC_COLLATE:                           en_US.UTF-8
>> LC_CTYPE:                             en_US.UTF-8
>
> you are at risk if the data is not legal UTF-8 strings.

But is it possible to store non-UTF-8 data in a UNICODE database?

> The real question therefore is whether you have the database encoding
> set correctly --- ie, is it UNICODE (== UTF8)?  If not then it may well
> be that Postgres is presenting strings to strcoll() that the latter 
> will
> choke on.

The database is UNICODE.

$ psql -U postgres -l
         List of databases
    Name    |  Owner   | Encoding
-----------+----------+-----------
  bric      | postgres | UNICODE
  template0 | postgres | SQL_ASCII
  template1 | postgres | SQL_ASCII
(3 rows)

I plan to dump it, run initdb with LC_COLLATE and LC_CTYPE both set to 
"C", and restore the database and see if that helps.

Thanks,

David

In response to

Responses

pgsql-general by date

Next:From: David WheelerDate: 2004-08-23 22:58:33
Subject: Re: UTF-8 and LIKE vs =
Previous:From: Markus BertheauDate: 2004-08-23 22:46:50
Subject: Re: UTF-8 and LIKE vs =

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group