Re: multi terabyte fulltext searching

On Wed, 21 Mar 2007, Benjamin Arai wrote:

> Can't you implement something similar to google by aggregating results for
> TSearch2 over many machines?

tsearch2 doesn't use any global statistics, so, in principle, you
should be able to run fts on several machines and combine them using
dblink (contrib/dblink).

>
> Benjamin
> On Mar 21, 2007, at 8:59 AM, Teodor Sigaev wrote:
>
>> I'm afraid that fulltext search on multiterabytes set of documents can not
>> be implemented on any RDBMS, at least on single box. Specialized fulltext
>> search engines (with exact matching and time to search about one second)
>> has practical limit near 20 millions of docs, cluster - near 100 millions.
>> Bigger collections require engines like a google.
>>
>>
>> --
>> Teodor Sigaev E-mail: teodor(at)sigaev(dot)ru
>> WWW:
>> http://www.sigaev.ru/
>>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83