Wrong results: NOT IN to anti-join with an upper-level Var in the sub-select's output

From: Rui Zhao <zhaorui126(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Wrong results: NOT IN to anti-join with an upper-level Var in the sub-select's output
Date: 2026-07-31 06:02:06
Message-ID: CAHWVJhGuaFFRpmq4j+mcMcm_HC5QOT7LZsC9bf9b7BCBmvbfMA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

383eb21eb converts a NOT IN to an anti-join once it can prove both the outer
expressions and the sub-select's output columns non-null. The second proof goes
wrong when an output column is an upper-level Var, and the query then keeps a
row it should have dropped. Column order matters here -- o.b and i.x have to
land on the same attnum:

\pset null '<NULL>'
CREATE TABLE o (b int, a int NOT NULL);
CREATE TABLE i (x int);
INSERT INTO o VALUES (NULL, 1);
INSERT INTO i VALUES (5);

SELECT * FROM o WHERE o.a NOT IN (SELECT o.b FROM i WHERE i.x IS NOT NULL);
b | a
--------+---
<NULL> | 1
(1 row)

The sub-select returns one row and its value is NULL, so the NOT IN is NULL and
that row should not come back:

SELECT o.a, o.b,
(SELECT count(*) FROM i WHERE i.x IS NOT NULL) AS subquery_rows,
(o.a NOT IN (SELECT o.b FROM i WHERE i.x IS NOT NULL)) AS
not_in_value
FROM o;
a | b | subquery_rows | not_in_value
---+--------+---------------+--------------
1 | <NULL> | 1 | <NULL>
(1 row)

and writing the same predicate so that it can't be pulled up agrees:

SELECT * FROM o
WHERE (o.a NOT IN (SELECT o.b FROM i WHERE i.x IS NOT NULL)) OR false;
b | a
---+---
(0 rows)

EXPLAIN (COSTS OFF) on the first query gets

Nested Loop Anti Join
Join Filter: (o.a = o.b)
-> Seq Scan on o
-> Materialize
-> Seq Scan on i
Filter: (x IS NOT NULL)

When expr_is_nonnullable() can't prove an output column non-null,
query_outputs_are_not_nullable() falls back to find_nonnullable_vars() over the
quals find_subquery_safe_quals() collected. find_nonnullable_vars() only
reports Vars of the current level, and the multibitmapset it returns identifies
them by varno and varattno alone, so the upper-level o.b -- varno 1, varattno 1
in the outer rangetable -- matches i.x, varno 1, varattno 1 in the sub-select's.
var_is_nonnullable() turns upper-level Vars away; this fallback doesn't:

- if (IsA(expr, Var))
+ if (IsA(expr, Var) && ((Var *) expr)->varlevelsup == 0)

There is nothing else to try for an upper-level Var there anyway, since
query_outputs_are_not_nullable() only has the sub-Query to work with. Patch
attached, with a test; the test fails without the one-line change, and make
check is green with it.

383eb21eb is in 19beta1, so this wants fixing on that branch too.

The same weakness has a second instance in the follow-up patch on [1], which
adds an outer-side proof with its own find_nonnullable_vars() lookup; Ayush
reported that one there [2]. reduce_outer_joins() also calls
find_nonnullable_vars(), but it only compares two such sets, both built at the
same level, so it isn't affected -- those two are the places that match a single
Var against the set.

[1] https://postgr.es/m/CAMbWs4_TNUs1jn7q0J-%3DEsz7ziiFdjDAtW4x2u6tv6H5hhmhDA%40mail.gmail.com
[2] https://postgr.es/m/CAJTYsWWH6Pm2xiq%3DOM3vnBEBz5%3DJp93G5_VUSKKGdkXinbXzZw%40mail.gmail.com

Thanks,
Rui

Attachment Content-Type Size
v1-0001-notin-subselect-upper-level-output-var.patch.txt text/plain 5.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message shveta malik 2026-07-31 06:10:43 Re: Support EXCEPT for TABLES IN SCHEMA publications
Previous Message vignesh C 2026-07-31 05:35:51 Re: sequencesync worker race with REFRESH SEQUENCES