| From: | PG Bug reporting form <noreply(at)postgresql(dot)org> |
|---|---|
| To: | pgsql-bugs(at)lists(dot)postgresql(dot)org |
| Cc: | 125atishprak(at)gmail(dot)com |
| Subject: | BUG #19854: GiST distance ordering fails on NaN coordinates, Assert in computeDistance |
| Date: | 2026-11-09 12:53:06 |
| Message-ID: | 19854-c255ec23cf0cb87c@postgresql.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-bugs |
The following bug has been logged on the website:
Bug reference: 19854
Logged by: Atish Prakash
Email address: 125atishprak(at)gmail(dot)com
PostgreSQL version: 18.6
Operating system: Linux x86_64
Description:
Ordering a GiST index scan by distance fails when NaN coordinates are
involved. On an
assert-enabled build, one row with a NaN coordinate takes the server through
crash
recovery. These values are accepted by the input functions.
1. NaN in a stored box:
CREATE TABLE b(c box);
INSERT INTO b VALUES ('(NaN,0),(1,1)');
INSERT INTO b SELECT '(5,5),(6,6)' FROM generate_series(1, 5);
CREATE INDEX ON b USING gist (c);
SET enable_seqscan = off;
SET enable_sort = off;
SELECT count(*) FROM (SELECT c FROM b ORDER BY c <-> '(0,0)'::point) s;
Assert-enabled build:
TRAP: failed Assert("box->low.x <= box->high.x"), File: "gistproc.c",
Line: 1253
Without assertions (19rc1): returns 6.
Polygons and circles with a NaN coordinate crash the assert build the
same way.
Without assertions (19rc1), they fail instead with:
ERROR: inconsistent point values
2. NaN in the query point, with ordinary stored values:
CREATE TABLE p(c polygon);
INSERT INTO p SELECT '((5,5),(6,5),(6,6))' FROM generate_series(1, 5);
CREATE INDEX ON p USING gist (c);
SET enable_seqscan = off;
SET enable_sort = off;
SELECT count(*) FROM (SELECT c FROM p ORDER BY c <-> '(NaN,NaN)'::point)
s;
ERROR: index returned tuples in wrong order
This happens with and without assertions (tested with assertions on 18.6,
19rc1 and
master, and without on 19rc1). The query point is an ordinary parameter
here.
Expected: no assertion failure or internal error, whatever order NaN sorts
in.
computeDistance() in gistproc.c asserts that a box's low bound does not
exceed its high
bound, which does not hold when a bound is NaN, since its comparisons are
all false.
An AddressSanitizer build without assertions reported no memory errors in
these cases,
so this looks like a robustness problem and not a memory-safety one.
NaN handling in the geometric code has come up before, for example in
"Strange behavior
with polygon and NaN":
https://www.postgresql.org/message-id/CAGf+fX70rWFOk5cd00uMfa__0yP+vtQg5ck7c2Onb-Yczp0URA@mail.gmail.com
I did not find this assertion reported. If this is considered part of that
known
problem, please treat this as one more example of it.
I have not written a patch.
Found by comparing index and sequential scans across the operator classes
listed in the
catalog. I reproduced and checked the result myself.
| From | Date | Subject | |
|---|---|---|---|
| Next Message | PG Bug reporting form | 2026-11-09 12:55:21 | BUG #19855: GiST index on multirange misses rows for <@ with a multirange query |
| Previous Message | Zhijie Hou | 2026-11-09 11:46:59 | Re: BUG #19839: Assert in ReorderBufferRestoreCleanup for aborted, spilled transaction |