From 79d8d82023c951086b484a52e85524c48625536c Mon Sep 17 00:00:00 2001
From: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu, 2 Jul 2026 17:19:30 -0400
Subject: [PATCH v3 2/5] Fix btree_gist's NotEqual strategy on internal index
 pages.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

gbt_var_consistent() handled the <> (BtreeGistNotEqual) strategy without
distinguishing leaf from internal pages, unlike every other strategy.
In particular, it tried to apply the datatype-specific f_eq method,
which is completely wrong since internal keys might not have the same
representation as leaf keys.  This led to OOB reads and potentially
crashes, and most likely to wrong query results as well.

On leaf pages we can apply the inverse of what the Equal strategy does.
On internal pages, just descend unconditionally: there's no useful way
to prove that a leaf page can't contain any unequal values.  (Maybe we
could do something in the case where upper and lower bounds are equal,
which is what the previous coding seemed to be thinking about.  But
that case is unlikely, and anyway it won't work for truncated bounds.)

Reported-by: 王跃林 <violin0613@tju.edu.cn>
Author: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/AH*AvQCYKhQGVvPWi1GiU4oY.8.1781609375063.Hmail.3020001251@tju.edu.cn
Backpatch-through: 14
---
 contrib/btree_gist/btree_utils_var.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/contrib/btree_gist/btree_utils_var.c b/contrib/btree_gist/btree_utils_var.c
index bceadf5ae60..99919ca3974 100644
--- a/contrib/btree_gist/btree_utils_var.c
+++ b/contrib/btree_gist/btree_utils_var.c
@@ -594,6 +594,13 @@ gbt_var_consistent(GBT_VARKEY_R *key,
 {
 	bool		retval = false;
 
+	/*
+	 * Remember that f_cmp is for internal pages, f_eq etc for leaf pages, and
+	 * on internal pages we need to check gbt_var_node_pf_match too.
+	 *
+	 * The leaf-page tests use swapped operands (e.g., f_gt(query, lower)
+	 * means "lower < query"), which is why they look reversed.
+	 */
 	switch (strategy)
 	{
 		case BTLessEqualStrategyNumber:
@@ -634,8 +641,13 @@ gbt_var_consistent(GBT_VARKEY_R *key,
 					|| gbt_var_node_pf_match(key, query, tinfo);
 			break;
 		case BtreeGistNotEqualStrategyNumber:
-			retval = !(tinfo->f_eq(query, key->lower, collation, flinfo) &&
-					   tinfo->f_eq(query, key->upper, collation, flinfo));
+			if (is_leaf)
+				retval = !(tinfo->f_eq(query, key->lower, collation, flinfo));
+			else
+			{
+				/* we can never disprove existence of a not-equal entry below */
+				retval = true;
+			}
 			break;
 		default:
 			retval = false;
-- 
2.52.0

