Skip site navigation (1) Skip section navigation (2)

Re: BUG #1931: ILIKE and LIKE fails on Turkish locale

From: Victor Snezhko <snezhko(at)indorsoft(dot)ru>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Volkan YAZICI <yazicivo(at)ttnet(dot)net(dot)tr>, pgsql-bugs(at)postgresql(dot)org
Subject: Re: BUG #1931: ILIKE and LIKE fails on Turkish locale
Date: 2006-09-22 08:04:35
Message-ID: upsdoxr70.fsf@indorsoft.ru (view raw or flat)
Thread:
Lists: pgsql-bugspgsql-tr-genel
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Victor Snezhko <snezhko(at)indorsoft(dot)ru> writes:
>> 2) When I try to create a stored procedure to create table (e.g., for
>>    creating table only if it doesn't already exist), it fails to
>>    compile if i use cyrillic letter "х" (unicode id: 0x0445, utf-8
>>    representation is D1 85) with the following weird error:
>
>> ERROR:  expected dot between identifiers: х
>> CONTEXT:  compile of PL/pgSQL function "createoraltertable" near line 2
>
>> the following query fails:
>
>> CREATE OR REPLACE FUNCTION TestFunction()
>> RETURNS int
>> AS $$
>> BEGIN
>>   SELECT х FROM test_table;
>>   RETURN 0;
>> END;
>> $$ LANGUAGE plpgsql;
>
> I tried to duplicate this without success on Fedora Core 5.  Either your
> ru_RU.utf8 locale works differently from Fedora's, or something else is
> wrong.  
> Looking at the code, the only very plausible theory is that
> isspace() is doing something different than we expect it to.  I wonder
> whether you have the database encoding set to something else than what
> the postmaster's LC_CTYPE locale expects?

No, the database encoding is UTF-8, the cluster was built with
ru_RU.UTF-8 locale, and I haven't modified any defaults.

It's on FreeBSD 6.1, and it's quite possible that it has something
wrong with utf-8 locale, I saw some complaints at least about
collation.

However, isspace doesn't seem like a culprit to me. Let's ensure that
I have done everything right to come to this conclusion. I have
applied the following patch to pl_funcs.c:


But, if I'm doing everything right, isspace seems not to be the
culprit to me. I have run the above query on the server with the
following applied patch (the server is 8.1.4, but I don't see any
changes to the plpgsql_convert_ident() in revs 1.46-1.54):

*** src/pl/plpgsql/src/pl_funcs.c.orig  Wed Nov 23 00:23:30 2005
--- src/pl/plpgsql/src/pl_funcs.c       Fri Sep 22 14:48:09 2006
***************
*** 363,368 ****
--- 363,381 ----
  {
        const char *sstart = s;
        int                     identctr = 0;
+       char                    *isspacebuf;
+       int                     i;
+
+       isspacebuf = (char*)malloc(strlen(s) + 1);
+       for (i=0; s[i]; ++i) {
+               if (isspace(s[i]))
+                       isspacebuf[i] = 't';
+               else
+                       isspacebuf[i] = 'f';
+       }
+
+       ereport(LOG,
+               (errmsg("plpgsql_convert_ident: ident=%s, isspace array=%s, isspace(0)=%c", s, isspacebuf, isspace(0)?'t':'f')));

        /* Outer loop over identifiers */
        while (*s)


and got the following line in the logfile:

  LOG:  plpgsql_convert_ident: ident=х, isspace array=ff, isspace(0)=f

The symbol after "input=" is that 0x0445 (D1 85 in utf-8) letter which
I mentioned above. I also double-checked this: got a stack trace (by
inserting NULL-pointer assignment instead of reporting "expected dot
between identifiers") and saw that the value passed to the
plpgsql_convert_ident points to the following bytes: D1 85 00.

So, isspace returns false on all three of them.

I have also run the same query on another cluster (and the same
binaries), initialized with a single-byte encoding: ru_RU.KOI8-R,
where everything works. isspace returns the same results:

LOG:  plpgsql_convert_ident: ident=х, isspace array=f, isspace(0)=f
LOG:  plpgsql_convert_ident: ident=test_table, isspace array=ffffffffff, isspace(0)=f

Strange...

-- 
WBR, Victor V. Snezhko
E-mail: snezhko(at)indorsoft(dot)ru



In response to

Responses

pgsql-tr-genel by date

Next:From: Victor SnezhkoDate: 2006-09-22 09:55:44
Subject: Re: BUG #1931: ILIKE and LIKE fails on Turkish locale
Previous:From: Tom LaneDate: 2006-09-21 15:50:38
Subject: Re: BUG #1931: ILIKE and LIKE fails on Turkish locale

pgsql-bugs by date

Next:From: Massimiliano CitterioDate: 2006-09-22 08:20:43
Subject: BUG #2644: pgadmin III foreign key
Previous:From: Christoph ZwerschkeDate: 2006-09-21 22:38:33
Subject: Re: BUG #2642: Connection problems without IPv6

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group