Re: Add CASEFOLD() function.

On 17/06/2025 20:14, Jeff Davis wrote:
> On Tue, 2025-06-17 at 17:37 +0200, Vik Fearing wrote:
>> If the character set of <character factor> is UTF8, UTF16, or UTF32,
>> then FR is replaced by
>>      Case:
>>          i) If the <search condition> S IS NORMALIZED evaluates to
>> True, then NORMALIZE (FR)
>>          ii) Otherwise, FR.
> I read that as "if the input is normalized, then the output should be
> normalized", IOW preserve the normalization. But does it mean "preserve
> whatever the input normal form is" or "preserve NFC if the input is
> NFC, otherwise the normalization is undefined"?
>
> The above wording seems to mean "preserve NFC if the input is NFC",
> because that's what NORMALIZE(FR) does when the normal form is
> unspecified.

Yes, and that is also the default for <normalized predicate>.

>> It does not appear to me that our LOWER and UPPER functions obey this
>> rule,
> You are correct:
>
> WITH s(t) AS
> (SELECT NORMALIZE(U&'\00C1\00DF\0301' COLLATE "en-US-x-icu"))
> SELECT UPPER(t) = NORMALIZE(UPPER(t)) FROM s;
> ?column?
> ----------
> f
>
>> so there is a valid argument that we should continue to ignore it.
>> Or, we can say that we have at least one of three compliant.
> What do other databases do?

I don't know.  I am just pointing out what the Standard says.  I think
we should either comply, or say that we don't do it for LOWER and UPPER
so let's keep things implementation-consistent.

> Given how costly normalization can be, imposing that on every caller
> seems like a bit much.

How much does it cost to check for NFC?  I honestly don't know the
answer to that question, but that is the only case where we need to
maintain normalization.

> And favoring NFC for the user unconditionally
> might not be the best thing. Then again, NFC is good most of the time,
> and there are patches to speed up normalization.

It's not unconditionally, it's only if the input was NFC.

> I tend to think that a lot of users who want casefolding would also
> want normalization, but it's hard to weigh that against the performance
> cost. It might not matter outside of a few edge cases, though I'm not
> sure exactly how many.

I defer to you and others in the thread to make this decision.

--

Vik Fearing