[PG19] Wrong results from Memoize with a nondeterministic collation

From: shihao zhong <zhong950419(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: David Rowley <dgrowleyml(at)gmail(dot)com>, Guofenglinux <guofenglinux(at)gmail(dot)com>
Subject: [PG19] Wrong results from Memoize with a nondeterministic collation
Date: 2026-10-08 03:33:31
Message-ID: CAGRkXqTBsBegZ8rNMYjwkeu1V5wRQ06zR6jTVickqP_3E7mb8g@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

I used Opus to analyze the new features in PG19, and it found a query
that 18 gets right and 19 gets wrong.

CREATE COLLATION case_insensitive (provider = icu,
locale = 'und-u-ks-level2', deterministic = false);
CREATE TABLE orders (code text COLLATE case_insensitive);
CREATE TABLE products (code text COLLATE "C" PRIMARY KEY);
INSERT INTO orders
SELECT CASE WHEN g % 2 = 1 THEN 'p1' ELSE 'P1' END
FROM generate_series(1, 2000) g;
INSERT INTO products
SELECT 'p' || g FROM generate_series(1, 50000) g;
ANALYZE orders, products;

SELECT count(*) FROM orders o WHERE NOT EXISTS
(SELECT 1 FROM products p WHERE p.code COLLATE "C" = o.code);

18 gives 1000, the 'P1' orders. 19 gives 0.

Since 0da29e4cb16 an anti join can use Memoize. Memoize compares its
cache key with the collation of o.code, so 'P1' hits the cache entry
of 'p1'. The clause compares under "C", where they differ.

The same clause in a LEFT JOIN is wrong on 14 and up, there the 'P1'
orders come out joined to 'p1'. So it is an old bug with a new way
in. It needs a nondeterministic collation and a COLLATE on the inner
side of the clause, so I doubt many people hit it. The attached
memoize-collation.sql runs both queries.

0001 labels the cache key with the clause's input collation in
paraminfo_get_equal_hashops(), like d237a7a8366 did for a semijoin's
RHS.Inner joins already get this from equivalence classes. EXPLAIN
output stays the same. Switching to binary mode on a mismatch would
also work, but it changes the cache mode of plans that are right
today. 0002 is a test, it is optional.

0001 applies to 17 through 19. On 14 to 16 the hunk has to be placed
by hand. The NOT EXISTS case is a regression from 18, since anti
joins could not use Memoize there. It is a narrow one, so I will
leave the open items list to you.

0002 is a test only patch.

Thanks,
Shihao

Attachment Content-Type Size
memoize-collation.sql application/octet-stream 2.1 KB
v1-0001-Use-the-join-clause-s-collation-for-Memoize-cache.patch application/octet-stream 1.4 KB
v1-0002-Add-a-test-for-Memoize-with-mismatched-collations.patch application/octet-stream 3.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrey Borodin 2026-10-08 03:53:17 Re: Compression of bigger WAL records
Previous Message shihao zhong 2026-10-08 03:31:49 Re: REPACK (CONCURRENTLY) can't complete after ~105M concurrent updates/deletes