Re: string_to_array eats too much memory?

From: Teodor Sigaev <teodor(at)sigaev(dot)ru>
To: Tatsuo Ishii <ishii(at)sraoss(dot)co(dot)jp>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: string_to_array eats too much memory?
Date: 2006-11-08 08:53:02
Message-ID: 45519AEE.8040704@sigaev.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> I'm playing with GIN to make a full text search system. GIN comes with
> built-in TEXT[] support and I use string_to_array() to make a
> TEXT[]. Problem is, if there's large number of array elemets,
> string_to_array() consumes too much memory. For example, to make ~70k
> array elements, string_to_array seems to eat several Gig bytes of
> memory. ~70k array elements means there are same number of words in a
> document which is not too big in a large text IMO.

Do you mean 70k unique lexemes? Ugh.
Why do not you use tsearch framework?

--
Teodor Sigaev E-mail: teodor(at)sigaev(dot)ru
WWW: http://www.sigaev.ru/

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Teodor Sigaev 2006-11-08 09:28:01 Re: Tsearch Index Size and GiST vs. GIN
Previous Message Teodor Sigaev 2006-11-08 08:43:18 Re: Index ignored with "is not distinct from", 8.2 beta2