gist index build produces corrupt result on first access to table.

From: Eric Davies <eric(at)barrodale(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: gist index build produces corrupt result on first access to table.
Date: 2004-03-03 19:34:13
Message-ID: 5.2.1.1.0.20040303092437.025db698@marilyn.barrodale.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

We've implemented a 5D box data type and have implemented both RTree and
GiST access methods under PostgresSQL 7.4 and PostgresSQL 7.4.1. The 5D box
internally looks like:
struct Box5D{
float minBounds[5];
float maxBounds[5];
};
and so takes up 40 bytes and is of fixed length. The GiST access methods
are basically a generalization of the 2D box indexing access methods from
Refraction Research's PostGIS.

We've tested this by building an index on a table that contains 30000 rows
(no nulls) representing small non-overlapping boxes on a 2D plane (ie, all
the values for indices 0 and 1 vary; the values for indices 2,3, and 4 are
constant). Then we try a select on the table with an overlaps test in the
where clause that should result in 66 rows being returned.
We've used an "explain select ..... " to verify that the index is used
during the search.

An RTree index built on the table works fine, always. The ability to build
a GiST index is less reliable; sometimes the index is built properly and
subsequent searchs on it work perfectly. Sometimes the index is built
incorrectly, and subsequent searchs on it fail. We have examined the keys
passed to our GiST consistent access method during the select and have
observed that when the index hasn't been built properly, the first key
passed to the consistent method is gibberish (strange values for indices 0
and 1, strange nonzero values for indices 2,3 and 4). When the index has
been built properly, the first key passed to the consistent method has
reasonable looking values.

We've been able to isolate this behavior somewhat: the index build only
seems to produce corrupt results if the data connection that sent the
command did not perform any preceding statements that would have caused the
boxes to have been read from disk. For example, the sequence:
<open connection to db>
create index box_index on box_table using gist(box_column);

would produce a corrupt index.
However, the sequence

<open connection to db>
select count(*) from box_table where box_column && '0 0 0 0 0 1 1 1 1 1';
create index box_index on box_table using gist(box_column);

produces a valid index,
as does the sequence:

<open connection to db>
create index box_index on box_table using gist(box_column);
drop index box_index;
create index box_index on box_table using gist(box_column);

In short, it appears that an operation that forces the boxes to read from a
table
needs to be done to "prime the index building pump".

We've tried installing PostGIS and it does not seem to be affected by any
similar problems.
The notable differences between our code and Refraction's are:
* our input geometries are fixed length, rather than variable length
* our compress method just returns the GISTENTRY * it was passed as an
argument. We've tried returning a deep copy of the GISENTRY *, but that
made no difference.
We've looked at the contrib/cube code, but it's also a variable length
parameter, and it uses a very different parameter passing mechanism.

In the short term, we can work around the issue, but it would be nice to be
able to fix the problem. Any ideas are welcome.

Thank you.
Eric.

**********************************************
Eric Davies, M.Sc.
Barrodale Computing Services Ltd.
Tel: (250) 472-4372 Fax: (250) 472-4373
Web: http://www.barrodale.com
Email: eric(at)barrodale(dot)com
**********************************************
Mailing Address:
P.O. Box 3075 STN CSC
Victoria BC Canada V8W 3W2

Shipping Address:
Hut R, McKenzie Avenue
University of Victoria
Victoria BC Canada V8W 3W2
**********************************************

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Bruce Momjian 2004-03-03 19:40:15 Re: libpq API for PQcmdTuples()
Previous Message scott.marlowe 2004-03-03 18:52:26 Re: How index are running and how to optimise ?