| From: | Rui Zhao <zhaorui126(at)gmail(dot)com> |
|---|---|
| To: | Andrew Dunstan <andrew(at)dunslane(dot)net> |
| Cc: | Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com>, Tristan Partin <tristan(at)partin(dot)io>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Add ASCII fast path to Unicode normalization functions |
| Date: | 2026-10-04 16:42:02 |
| Message-ID: | CAHWVJhHPOiD+By2pEmn=uDw5Cj5-3vyfXXvS84zDysnETwqPpQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi Andrew,
Thanks for working on this. I tried two further optimizations on top
of v4. They can be applied independently:
1. unicode_assigned() still calls pg_mbstrlen_with_len() on the
remaining input, then walks it again to check each code point. 0001
uses the byte length to walk to the end directly. This also avoids
scanning the whole suffix before returning false for an unassigned
code point near the beginning.
2. unicode_normalize_func() and unicode_is_normalized() allocate a C
string for the normalization form on every call, even when the input
is all ASCII. 0002 compares the form directly in the text value,
preserving case-insensitive matching and rejecting extra bytes.
It only builds a C string when reporting an invalid form.
0003 replaces the constant-folded VALUES test with stored short and
compressed text values.
Core regression and the Unicode normalization checks passed with these
patches.
Regards,
Rui
| Attachment | Content-Type | Size |
|---|---|---|
| nocfbot-0001-Avoid-counting-code-points-before-checking-Unicode-a.patch | application/octet-stream | 1.5 KB |
| nocfbot-0002-Avoid-allocating-a-C-string-for-the-normalization-fo.patch | application/octet-stream | 5.2 KB |
| nocfbot-0003-Exercise-ASCII-normalization-with-stored-text-values.patch | application/octet-stream | 2.6 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Alexander Korotkov | 2026-10-04 17:36:34 | Re: Asynchronous MergeAppend |
| Previous Message | Stefan Guha | 2026-10-04 16:24:49 | Planning time quadratic in the IN-list length for "c = X AND (a, b) IN (...)" with BitmapOr |