Incorrect behaviour when using a GiST index on points

From: Alexander Korotkov <aekorotkov(at)gmail(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Incorrect behaviour when using a GiST index on points
Date: 2012-02-16 07:43:11
Message-ID: CAPpHfdvGticGniaj88VCHzHboXJobUhjLm6c09q_Op_u9EoBFg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi!

Oleg and me found an incorrect behaviour when using a GiST index on points.
It's quite easy to reproduce.

test=# create table test as values (0,0.0000001);
test=# insert into test values (0,0.0000001);
test=# select * from test where point(x,y) <@ box(point(0,0),point(0,0));
x | y
---+---
(0 rows)

test=# create index gist_test on test using gist (point(x,y));
test=# set enable_seqscan = off;
test=# select * from test where point(x,y) <@ box(point(0,0),point(0,0));
x | y
---+-------
0 | 1e-07
(1 row)

So, "point <@ box" operator don't thinks (0, 1e-07) to be equal to (0, 0),
while GiST index does. "point <@ box" operator uses on_ob function which
uses comparison operators of float numbers.

Datum
on_pb(PG_FUNCTION_ARGS)
{
Point *pt = PG_GETARG_POINT_P(0);
BOX *box = PG_GETARG_BOX_P(1);

PG_RETURN_BOOL(pt->x <= box->high.x && pt->x >= box->low.x &&
pt->y <= box->high.y && pt->y >= box->low.y);
}

GiST consistent method uses box_contained function, which uses FP* macros
for float comparison.

/* box_contained - is box1 contained by box2?
*/
Datum
box_contained(PG_FUNCTION_ARGS)
{
BOX *box1 = PG_GETARG_BOX_P(0);
BOX *box2 = PG_GETARG_BOX_P(1);

PG_RETURN_BOOL(FPle(box1->high.x, box2->high.x) &&
FPge(box1->low.x, box2->low.x) &&
FPle(box1->high.y, box2->high.y) &&
FPge(box1->low.y, box2->low.y));
}

Correspondingly, FP* compares numbers with some error.

#define EPSILON 1.0E-06

#ifdef EPSILON
#define FPzero(A) (fabs(A) <= EPSILON)
#define FPeq(A,B) (fabs((A) - (B)) <= EPSILON)
#define FPne(A,B) (fabs((A) - (B)) > EPSILON)
#define FPlt(A,B) ((B) - (A) > EPSILON)
#define FPle(A,B) ((A) - (B) <= EPSILON)
#define FPgt(A,B) ((A) - (B) > EPSILON)
#define FPge(A,B) ((B) - (A) <= EPSILON)
#else
#define FPzero(A) ((A) == 0)
#define FPeq(A,B) ((A) == (B))
#define FPne(A,B) ((A) != (B))
#define FPlt(A,B) ((A) < (B))
#define FPle(A,B) ((A) <= (B))
#define FPgt(A,B) ((A) > (B))
#define FPge(A,B) ((A) >= (B))
#endif

Described differences leads to incorrect behaviour of GiST index.
The question is: what is correct way to fix it? Should on_pb also use FP*
or consistent method should behave like on_pb?

------
With best regards,
Alexander Korotkov.

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Marko Kreen 2012-02-16 08:17:21 Re: Speed dblink using alternate libpq tuple storage
Previous Message Pavel Golub 2012-02-16 06:51:17 Re: Google Summer of Code? Call for mentors.