Re: tsearch2 poor performance

Kris,

is't possible to share your "paragraph generator" ? It'd be better than
downloading the whole dump :)

On Wed, 29 Sep 2004, Kris Kiger wrote:

> Oleg, the data I have right now was generated using a random paragraph
> generator. The words are real words, but there are only 508 distinct
> keywords in the 3,000,000 records that tsearch2 will pick up, using
> default settings. I was using this data set for the purpose of testing
> tsearch2's capabilities, so it's not real world data. If you still want
> it, let me know where to send it and I will send you a dump of the DB.
>
> Kris
>
>
>
> Oleg Bartunov wrote:
>
> >Kris,
> >
> >we're working on prototype of tsearchd - full text search daemon, which
> >maintain static inverted index outside of postgresql using the same
> >parser, dictionary tsearch2 does. This approach could scale up
> >fts capability preserving access to metadata, so yo may have
> >"archive" part of your collection (tsearchd) and "online", which could be
> >searchable with tsearch2.
> >
> >Here is what we have right now:
> >
> >pages ( tid integer, fts_index tsvector)
> >
> >1. Create index
> >select count(tdindex(tid,fts_index)) from pages;
> >2. Flush index
> >select tdflush();
> >3. Search
> >select pages.tid, rank(fts_index,to_tsquery('supernovae & magellan')) as rank
> >from pages, tdsearch(to_tsquery('supernovae & magellan')) as idx where
> >tid=idx order by rank desc;
> >
> >If it's possible, you could share your data, so we could test our
> >prototype on real data.
> >
> >
> > Oleg
> >
> >
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83