Re: [HACKERS] indexable and locale

Here is Tom's comment on the patch.

> Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp> writes:
> >> Attached is a patch to the old problem discussed feverly before 6.5.
>
> > ... I think your pacthes break
> > non-ascii multi-byte character sets data and should be surrounded by
> > #ifdef LOCALE rather than replacing current codes surrounded by
> > #ifndef LOCALE.
>
> I am worried about this patch too. Under MULTIBYTE could it
> generate invalid characters? Also, do all non-ASCII locales sort
> codes 0-126 in the same order as ASCII? I didn't think they do,
> but I'm not an expert.
>
> The approach I was considering for fixing the problem was to use a
> loop that would repeatedly try to generate a string greater than the
> prefix string. The basic loop step would increment the rightmost
> byte as Goran has done (or, if it's already up to the limit, chop
> it off and increment the next character position). Then test to
> see whether the '<' operator actually believes the result is
> greater than the given prefix, and repeat if not. This avoids making
> any strong assumptions about the sort order of different character
> codes. However, there are two significant issues that would have
> to be surmounted to make it work reliably:
>
> 1. In MULTIBYTE mode incrementing the rightmost byte might yield
> an illegal multibyte character. Some way to prevent or detect this
> would be needed, lest it confuse the comparison operator. I think
> we have some multibyte routines that could be used to check for
> a valid result, but I haven't looked into it.
>
> 2. I think there are some locales out there that have context-
> sensitive sorting rules, ie, a given character string may sort
> differently than you'd expect from considering the characters in
> isolation. For example, in German isn't "ss" treated specially?
> If "pqrss" does not sort between "pqrs" and "pqrt" then the entire
> premise of *both* sides of the LIKE optimization falls apart,
> because you can't be sure what will happen when comparing a prefix
> string like "pqrs" against longer strings from the database.
> I do not know if this is really a problem, nor what we could do
> to avoid it if it is.
>
> regards, tom lane
>
> ************
>

--
Bruce Momjian | http://www.op.net/~candle
maillist(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026