Re: Add ASCII fast path to Unicode normalization functions

From: Rui Zhao <zhaorui126(at)gmail(dot)com>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com>, Tristan Partin <tristan(at)partin(dot)io>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Add ASCII fast path to Unicode normalization functions
Date: 2026-10-04 16:42:02
Message-ID: CAHWVJhHPOiD+By2pEmn=uDw5Cj5-3vyfXXvS84zDysnETwqPpQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Andrew,

Thanks for working on this. I tried two further optimizations on top
of v4. They can be applied independently:

1. unicode_assigned() still calls pg_mbstrlen_with_len() on the
remaining input, then walks it again to check each code point. 0001
uses the byte length to walk to the end directly. This also avoids
scanning the whole suffix before returning false for an unassigned
code point near the beginning.

2. unicode_normalize_func() and unicode_is_normalized() allocate a C
string for the normalization form on every call, even when the input
is all ASCII. 0002 compares the form directly in the text value,
preserving case-insensitive matching and rejecting extra bytes.
It only builds a C string when reporting an invalid form.

0003 replaces the constant-folded VALUES test with stored short and
compressed text values.

Core regression and the Unicode normalization checks passed with these
patches.

Regards,
Rui

Attachment Content-Type Size
nocfbot-0001-Avoid-counting-code-points-before-checking-Unicode-a.patch application/octet-stream 1.5 KB
nocfbot-0002-Avoid-allocating-a-C-string-for-the-normalization-fo.patch application/octet-stream 5.2 KB
nocfbot-0003-Exercise-ASCII-normalization-with-stored-text-values.patch application/octet-stream 2.6 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Alexander Korotkov 2026-10-04 17:36:34 Re: Asynchronous MergeAppend
Previous Message Stefan Guha 2026-10-04 16:24:49 Planning time quadratic in the IN-list length for "c = X AND (a, b) IN (...)" with BitmapOr