Re: locale / encoding / meson cleanup

On 2026-08-23 Su 9:00 AM, Andrew Dunstan wrote:
> Hi,
>
> (Astute observers will notice that Claude loves the sound of its own
> voice - or my voice which it's trying to emulate - a lot more than I
> do. I have trimmed the text quite a bit.)
>
> Tom observed in [1] that nothing in the buildfarm builds a cluster
> with locale C and encoding UTF8, that this is where the recent
> to_date() crash went undetected, and that the animal configuration had
> no way to ask for one. I've taught the buildfarm client to accept an
> encoding alongside the locale. However, it's not yet released, because
> running an animal that way turned up two things, and the second
> explains in part why the first went unnoticed for as long as it did.
>
> 1. test_regex_utf8 depends on the ctype, not just the encoding
>
> The file decides whether to run by looking at the encoding alone:
>
>     SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset
>
> but two of its cases also depend on the database ctype, so it fails in a
> database with encoding UTF8 and locale C:
>
>     @@ -152,7 +152,7 @@
>         test_regex
>      -----------------
>       {0,REG_ULOCALE}
>     - {xᔀሷ}
>     + {x}
>      (2 rows)
>
>      select * from test_regex('[[:lower:]]+',  E'xᔀሷ', 'L');
>     @@ -166,7 +166,7 @@
>         test_regex
>      -----------------
>       {0,REG_ULOCALE}
>     - {xᔀሷ}
>     + {x}
>      (2 rows)
>
> The new output is the correct one: under ctype C, isgraph() and
> isprint() are
> false for anything outside ASCII, so only the x matches. The cases are
> [[:graph:]] and [[:print:]] over E'xᔀሷ'. It isn't really about the
> regex code;
> the same difference shows up in plain SQL in two clusters differing
> only in
> locale.
>
> Those two are the only ctype-dependent assertions in the file —
> everything else
> uses explicit code point ranges such as [\u1000-\u2000], or an input
> with a
> separator (x*, x_*) that ends the match inside ASCII, which is presumably
> deliberate.
>
> Patches 0001 (for 15 and 16) and 0002 (for 17+) attached.
>
> 0002 gives the two cases an explicit collation:
>
>     select * from test_regex('[[:graph:]]+',  E'xᔀሷ' COLLATE
> pg_c_utf8, 'L');
>     select * from test_regex('[[:print:]]+',  E'xᔀሷ' COLLATE
> pg_c_utf8, 'L');
>
> test_regex.c already threads PG_GET_COLLATION() into the compile, and
> pg_c_utf8 exists in every UTF8 database, which is the only place this
> file
> runs. That returns {xᔀሷ} in a C+UTF8 database and in en_US.utf8, so
> the result
> lines in the expected file don't change at all — only the echoed query
> text
> does. It seems to me strictly better than what's there now, since the
> cases
> stop depending on how the animal happened to be initdb'd.
>
> On 15 and 16 there's no collation to point at — ucs_basic has
> collctype C, the builtin provider is 17+, and ICU depends on the build
> — so 0001 just adds a second expected file with the C-ctype answers,
> the way json_encoding.sql does for its two encodings.
>
> I did consider just neutralising the two inputs, by putting a space or
> a tab in
> front of the non-ASCII characters the way x* and x_* already do
> elsewhere in
> that block. It works and it backpatches everywhere. I'd be sorry to do
> it,
> though: those two cases are the only ones in the file that exercise
> Unicode
> ctype at all, and after such a change they would pass even if the
> ctype lookup
> for non-ASCII were completely broken.

I have pushed these two. That's enough to unblock the buildfarm work.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com