Re: index support for arrays (GiST)

From: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
To: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
Cc: selkovjr(at)mcs(dot)anl(dot)gov, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, vmikheev(at)SECTORBASE(dot)COM, "'pgsql-hackers '" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: index support for arrays (GiST)
Date: 2000-12-15 12:08:26
Message-ID: Pine.GSO.3.96.SK.1001215140503.20670O-100000@ra
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

Well,

we found an answer ourserlves. Patch for 7.0.3 is included below.
Credits to Teodor Sigaev (teodor(at)stack(dot)net)
Some comments:

>From src/backend/access/gist/gist.c
/*
** Take a compressed entry, and install it on a page. Since we now know
** where the entry will live, we decompress it and recompress it using
** that knowledge (some compression routines may want to fish around
** on the page, for example, or do something special for leaf nodes.)
*/

After compressing of index it's written to disk decompressed (!) which
is the reason we have the problem with broken index !
It looks like other people just didn't use index decompression function
(at least in Gene's code decompression function just do return ) and
that's why this bug was not discovered. We could make a patch for
upcoming 7.1 if hackers desired. I consider this patch as a bugfix
not a new feature or improvement. We got a very promising results.

Another question to this code is - why gistPageAddItem does
compress - decompress - compress. It's not clear from the comment.

Best regards,

Oleg

-------------------------------------------------------------------------
maze% diff -c backend/access/gist/gist.c
backend/access/gist/gist.c.orig
*** backend/access/gist/gist.c Fri Dec 15 13:03:40 2000
--- backend/access/gist/gist.c.orig Fri Dec 15 13:00:50 2000
***************
*** 374,380 ****
{
GISTENTRY tmpcentry;
IndexTuple itup = (IndexTuple) item;
- OffsetNumber retval;

/*
* recompress the item given that we now know the exact page and
--- 374,379 ----
***************
*** 386,400 ****
IndexTupleSize(itup) -
sizeof(IndexTupleData), FALSE);
gistcentryinit(giststate, &tmpcentry, dentry->pred, r, page,
offsetNumber, dentry->bytes, FALSE);
! *newtup = gist_tuple_replacekey(r, tmpcentry, itup);
! retval = PageAddItem(page, (Item) *newtup,
IndexTupleSize(*newtup),
! offsetNumber, flags);
/* be tidy */
if (tmpcentry.pred != dentry->pred
&& tmpcentry.pred != (((char *) itup) +
sizeof(IndexTupleData)))
pfree(tmpcentry.pred);

! return (retval);
}


--- 385,398 ----
IndexTupleSize(itup) -
sizeof(IndexTupleData), FALSE);
gistcentryinit(giststate, &tmpcentry, dentry->pred, r, page,
offsetNumber, dentry->bytes, FALSE);
! *newtup = gist_tuple_replacekey(r, *dentry, itup);
/* be tidy */
if (tmpcentry.pred != dentry->pred
&& tmpcentry.pred != (((char *) itup) +
sizeof(IndexTupleData)))
pfree(tmpcentry.pred);

! return (PageAddItem(page, (Item) *newtup,
IndexTupleSize(*newtup),
! offsetNumber, flags));
}

-----------------------------------------------------------------------

On Wed, 13 Dec 2000, Oleg Bartunov wrote:

> Date: Wed, 13 Dec 2000 18:48:40 +0300 (GMT)
> From: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
> To: selkovjr(at)mcs(dot)anl(dot)gov
> Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, vmikheev(at)SECTORBASE(dot)COM,
> 'pgsql-hackers ' <pgsql-hackers(at)postgresql(dot)org>
> Subject: [HACKERS] index support for arrays (GiST)
>
> Hi,
>
> we are getting a bit close to add index support for int arrays using
> GiST interface. This will really drive up performance of our full text
> search fully based on postgresql. We have a problem with broken index
> and couldn't find a reason. I attached archive with sources
> for GiST functions and test suite to show a problem - vacuum analyze
> at end end of TESTSQL should complain about broken index.
> Here is a short description:
> 1. untar in contrib 7.0.*
> 2. cd _intarray
> 3. edit Makefile for TESTDB (name of db for test)
> 4. createdb TESTDB
> 5. gmake
> 6. gmake install
> 7. psql TESTDB < TESTSQL
>
> Regards,
>
> Oleg
> _____________________________________________________________
> Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
> Sternberg Astronomical Institute, Moscow University (Russia)
> Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
> phone: +007(095)939-16-83, +007(095)939-23-83
>

_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Esa Pikkarainen 2000-12-15 12:14:45 Re: Sequence value
Previous Message Marco Wittwer 2000-12-15 11:58:29 How can I set the right of access in a pgsql database?

Browse pgsql-hackers by date

  From Date Subject
Next Message pgsql-bugs 2000-12-15 12:44:47 Outer joins aren't working with views
Previous Message Zeugswetter Andreas SB 2000-12-15 11:41:59 heap page corruption not easy