Re: ILIKE '%' never matches a non-ASCII byte in single-byte C-ctype databases

From: Manu <manuelreyesbravo(at)gmail(dot)com>
To: Greg Burd <greg(at)burd(dot)me>
Cc: Jeff Davis <pgsql(at)j-davis(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: ILIKE '%' never matches a non-ASCII byte in single-byte C-ctype databases
Date: 2026-10-18 17:13:11
Message-ID: 179234359102.298269.627069686706851202@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Greg,

> what do you think? bug, or misunderstanding on my part, or...? :)

Bug, and I think v1 is the right and complete fix. I reproduced it on x86-64
Linux with gcc, so another platform and compiler than yours: 'xÆy' ILIKE '%Æ%'
is t on 18.6 and f on 19rc1 and master (@45fc51fcbf4), in LATIN1, WIN1252 and
KOI8R, and master+v1 makes it t again. Your 016 test fails on master with the
same got: and passes with v1, and regression (239/239), pg_trgm and test_misc
all pass. master built with -funsigned-char (a stand-in for ARM/POWER) gives 0
differences from 18.6, which matches your note that those are unaffected.

I swept 1e493158d3d too: firstpat is the only place it left a signed char
compared against an unsigned char -- the other MatchText comparisons are
GETCHAR vs GETCHAR, and the non-MATCH_LOWER instantiations stay char vs char.
Targeted cases (escape \, ESCAPE, bytes 0x80 and 0xFF, NOT ILIKE, and the name
and bpchar types) all fail on 19 and v1 fixes every one; a 20,000-pair
differential fuzz against 18.6 and a separate oracle finds 1,540 differing
rows on 19, all ILIKE and none LIKE, and 0 with v1.

One scope point, maybe worth a word in the commit message or a test: the
trigger is a C collation on the expression, not a C-ctype database as such. In
any single-byte database, including one with a non-C locale,

SELECT 'xÆy' COLLATE "C" ILIKE '%Æ%';

returns t on 18.6 and f on 19. The same reaches name columns (which collate
"C"), an explicit COLLATE "C", and convert_from(), which inherits "C" from its
name argument. v1 fixes all of them, since they share the path.

The index side matches what you saw in pg_tre: like_fixed_prefix_ci is fine
(it keeps the Æ in the prefix), so a plain Æ% scan is correct, but a pattern
that rechecks (Æ%Æ, %Æ%) returns nothing on 19 because the recheck runs the
same ILIKE -- seq scan and index agree with each other, both empty. v1 fixes
those as well.

regards,
Manu

In response to

Browse pgsql-hackers by date

  From Date Subject
Previous Message Hannu Krosing 2026-10-18 16:48:50 [PATCH v1] Generic Datum Encoding and Compression Contexts (pg_encoding_type, pg_encoding_context)