ILIKE '%' never matches a non-ASCII byte in single-byte C-ctype databases

From: Greg Burd <greg(at)burd(dot)me>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Jeff Davis <pgsql(at)j-davis(dot)com>
Subject: ILIKE '%' never matches a non-ASCII byte in single-byte C-ctype databases
Date: 2026-10-18 15:30:47
Message-ID: _GW2V-rd0QW4BTguAJriA9XVmjOn2mR3-YeAZr9ZU6EtlLvYPEm_r_fV5wGKzSNMzboj_nqn3fk72AYAvOKzEnPlCCHteaYrgESrIhPg6l8=@burd.me
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Hackers, Jeff,

In v19, ILIKE in a database with a single-byte encoding and a C ctype
never matches when '%' is followed by a byte >= 0x80:

CREATE DATABASE l1 TEMPLATE template0 ENCODING 'LATIN1' LOCALE 'C';
\c l1
SET client_encoding = 'UTF8';
SELECT 'xÆy' ILIKE '%Æ%'; -- 18.6: t 19rc1: f
SELECT 'abÆ' ILIKE '%Æ'; -- 18.6: t 19rc1: f
SELECT 'Æcd' ILIKE 'Æ%'; -- t on both
SELECT 'xÆy' LIKE '%Æ%'; -- t on both

The same happens in WIN1252 and KOI8R databases with LOCALE 'C'. It
reproduces on 19rc1 and on master and REL_19_STABLE as of today.

The cause is in commit 1e493158d3d ("Remove char_tolower() API"). That
path is C_IMatchText, where like_match.c's GETCHAR() became
pg_ascii_tolower(), which returns unsigned char. The '%' branch of
MatchText keeps the first pattern byte in a char:

char firstpat;
...
firstpat = GETCHAR(*p);
...
if (GETCHAR(*t) == firstpat || nondeterministic)

For 0xC6, firstpat holds -58 where char is signed, while GETCHAR(*t)
promotes to 198, so the test never succeeds and the scan after '%' skips
every position. Before that commit GETCHAR was SB_lower_char(), which
returned char, so both sides had the same signedness. The other
comparisons in MatchText compare GETCHAR() with GETCHAR(), unsigned on
both sides, which is why a pattern without a leading '%' still matches.
Platforms where char is unsigned (ARM, POWER) are not affected.

The attached patch declares firstpat as int. That covers the
non-MATCH_LOWER instantiations too, where GETCHAR() is the plain char
and both sides are promoted from the same type.

The main regression database is normally UTF-8, which never reaches
C_IMatchText, so the patch adds a TAP test,
src/test/modules/test_misc/t/016_ilike_single_byte.pl, that creates a
LATIN1 / C database and checks seven ILIKE cases. It builds its
strings with convert_from() so it doesn't depend on the client
encoding. On master without the fix it fails:

got: 'false,false,false,false,true,true,false'
expected: 'true,true,false,true,true,false,false'

and with it, it passes, as do make check (239/239) and the rest of
test_misc (FreeBSD 15.1 amd64, clang 19; built and tested on master at
45fc51fcbf4).

I found this through pg_tre's regression suite, which compares its
index with a sequential scan in LATIN1, WIN1252 and KOI8R databases.
The ILIKE checks for words that start with a non-ASCII letter came back
empty on 19rc1 for both scans, because both end in core's ILIKE.

This should go into 19 before release; the patch carries
"Backpatch-through: 19".

what do you think? bug, or misunderstanding on my part, or...? :)

-greg

Attachment Content-Type Size
v1-0001-Fix-ILIKE-followed-by-a-non-ASCII-byte-in-single-byt.patch text/x-patch 4.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2026-10-18 15:45:56 Re: [DEPARSE] Pretty-printed sublink test expressions lose needed parentheses
Previous Message Jim Jones 2026-10-18 14:14:25 Re: Reject COPY TO of unlogged relations during recovery