Re: string_to_array eats too much memory?

> > I'm testing how GIN scales.
>
> Have a look at http://www.sigaev.ru/cvsweb/cvsweb.cgi/ftsbench/ - utility is
> specially developed for measuring performance of full-text solutions ( now it
> supports PgSQL( GiST, GIN ) and MySQL ). Right now I'm searching good query
> statistic for simulate load, but this data is a closed information in
> internet-wide search engines :(

Thanks.

> GIN itself is a just a tool for speedup searches, linguistic part is still in
> tsearch2.
>
> It's possible to use tsearch2 without any indexes at all. GiST and GIN is a way
> to speedup searches.
>
> Of course, you can develop another framework for full text search and framework
> may use GIN as it wish :)

Porblem with Japanese is, it's an agglutinative language and we need
to separate each word from a sentence. So, I need to modify tsearch2
anyway (I know someone from Japan is working on this).

BTW, can tsearch2 handle ~70k words in a document?
--
Tatsuo Ishii
SRA OSS, Inc. Japan