BUG #19641: Unexpected results on an SP-GiST indexed column with a non-deterministic collation

The following bug has been logged on the website:

Bug reference: 19641
Logged by: Suyang Zhong
Email address: syzhong16(at)gmail(dot)com
PostgreSQL version: 19beta3
Operating system: Ubuntu 22.04
Description:

Hi,

Consider the following test case:

CREATE COLLATION nd (provider = icu, locale = 'und-u-ks-level2',
deterministic = false);

CREATE TABLE t0(c1 text);
INSERT INTO t0 VALUES ('ALPHA');
CREATE INDEX ON t0 USING spgist (c1 COLLATE nd);

SELECT c1, c1 COLLATE nd = 'alpha' AS p FROM t0;
-- ALPHA | t

SET enable_seqscan = off;
SELECT count(*) FROM t0 WHERE c1 COLLATE nd = 'alpha';
-- Expected: 1, Actual: 0

RESET enable_seqscan;
SELECT count(*) FROM t0 WHERE c1 COLLATE nd = 'alpha';
-- 1

The predicate evaluates to true for the row, so filtering on the same
predicate should return it.

The original test case, where the planner chooses the index by itself:

CREATE TABLE t1(c1 text);
INSERT INTO t1
SELECT CASE WHEN i % 3 = 0 THEN 'alpha'
WHEN i % 3 = 1 THEN 'ALPHA'
ELSE 'beta' END
FROM generate_series(1, 340) AS i;
CREATE INDEX ON t1 USING spgist (c1 COLLATE nd);

SELECT count(*) FROM t1 WHERE c1 COLLATE nd = 'alpha';
-- Expected: 227, Actual: 113

With the v2 patch from #19633 applied, this case is still unchanged.
Reproduced on 20devel, 19beta3 and 18.6.