Re: BUG #19705: One NaN box makes a BRIN box_inclusion_ops index omit unrelated rows

Hi Kirill,

> But I cannot reproduce any wrong results with unpatched binary for
> KNN.

It does fail on master. Your points lie on one line and the query is at
the origin, so the low corner is always the closest point of a box. Try
a grid:

CREATE TABLE knn (p point);
INSERT INTO knn SELECT point(x, y)
FROM generate_series(0, 99) x, generate_series(0, 99) y;
INSERT INTO knn VALUES ('(NaN,NaN)');
CREATE INDEX ON knn USING gist (p);
SET enable_seqscan = off;
SELECT p FROM knn ORDER BY p <-> point '(99,99)' LIMIT 1;

Master returns (92,99), v5 returns (99,99). With a NaN high corner,
computeDistance() falls to the vertex case and returns the distance to
the low corner. That is too big for a query point inside the box. The
new union code alone fixes fresh indexes. Without the distance hunk, an
index built by master still gives the same wrong answers.

> So, v4 keeps point <@ polygon and point <@ circle losing NaN points
> that the heap returns.

0003 looks right to me, it just needs pgindent. Your circle test passes
without the circle hunk though, its failure on master comes from 0002.
A NaN point only matches a circle when it also has an infinity and the
radius is infinite.

The root of the polygon case is point_inside(), which puts (NaN,NaN)
inside every polygon, even '((0,0))'. Changing that changes query
results, so I'd leave it for a separate master only patch.

Thanks,
Shihao