Fixing the Turkish problem

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-hackers(at)postgreSQL(dot)org
Subject: Fixing the Turkish problem
Date: 2004-05-06 13:02:30
Message-ID: 26469.1083848550@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

We're sort of halfway there on coping with the Turkish-locale i-vs-I
problem. I'd like to finish the job for 7.5.

What we presently have is that identifier and keyword downcasing is done
without trusting tolower():

/*
* SQL99 specifies Unicode-aware case normalization, which we don't yet
* have the infrastructure for. Instead we use tolower() to provide a
* locale-aware translation. However, there are some locales where this
* is not right either (eg, Turkish may do strange things with 'i' and
* 'I'). Our current compromise is to use tolower() for characters with
* the high bit set, and use an ASCII-only downcasing for 7-bit
* characters.
*/
for (i = 0; i < len; i++)
{
unsigned char ch = (unsigned char) ident[i];

if (ch >= 'A' && ch <= 'Z')
ch += 'a' - 'A';
else if (ch >= 0x80 && isupper(ch))
ch = tolower(ch);
result[i] = (char) ch;
}

AFAICS the remaining problem is that there are a bunch of places that
use strcasecmp() or strncasecmp() to match inputs against locally known
keywords (such as datestyle or timezone names). We need to make a
variant version of strcasecmp that uses this same style of case-folding.

What I'm thinking of doing is inventing "pg_strcasecmp" and
"pg_strncasecmp" that act like the above and replacing all calls of the
standard library functions with these.

The routines need to be available in client code (eg, psql) as well as
the backend, so I'm thinking of putting them into libpgport (src/port/).
Another possibility would be to associate them with the multibyte
character code, which is already imported into client code in places.

Any thoughts, objections?

regards, tom lane

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2004-05-06 13:06:57 Re: Is there any method to keep table in memory at startup
Previous Message Tom Lane 2004-05-06 12:40:47 Re: Is there any method to keep table in memory at startup