Re: Add ASCII fast path to Unicode normalization functions

On 2026-10-03 Sa 3:41 PM, Ayush Tiwari wrote:
> Hi,
>
> On Sat, 3 Oct 2026 at 19:57, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>>
>> On 2026-09-18 Fr 10:11 AM, Andrew Dunstan wrote:
>>>
>>> When time permits I'll send a new version, incorporating your review
>>> and Bilal's suggestion.
>>>
>>>
>> Here's v3, rebased on current master. It folds in Bilal's change to pick
>> up where the ASCII scan stopped rather than decoding the whole string, and
>> deals with Tristan's review comments:
>>
>> - The helper is now valid_ascii_prefix_len(), which works on raw bytes
>> rather than a text value, and returns the length of the valid ASCII
>> prefix it found, or the full length if the whole string is valid ASCII.
>> That gets rid of the -1 return in v2, and settles on "valid ASCII"
>> throughout, as Tristan suggested.
>>
>> - The paragraph about Unicode normalization has moved out of the helper's
>> comment and into the callers, where it actually matters.
>>
>> - As in Bilal's patch, normalize() and IS NORMALIZED restart with the last
>> ASCII character before the non-ASCII part, since a following combining
>> mark can compose with it. ASCII characters have no decomposition and
>> are all starters, so nothing earlier can be affected, and the quick
>> check gives the same answer starting there as from the beginning of the
>> string.
>>
>> - There's a new regression test sweeping ASCII prefixes of 0 to 40 bytes
>> for normalize() (NFC and NFD) and unicode_assigned(), including a
>> combining mark right after the ASCII. It only lists lengths that give
>> the wrong answer, so the expected output is short. I checked that it
>> does fail if the last ASCII character isn't included in the part that
>> gets normalized.
>>
>> I'll add it to the next CF.
> I see CI tests failing for this patch. I haven't gone through the patch
> yet, but from the logs it looks related to missing trailing spaces in
> the newly added expected-output headings. The pg_upgrade failure seems
> to be in its regression setup, not the upgrade itself.

Sorry about that. Here's a v4 that should have the whitespace issue fixed.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com