From: | Ildus Kurbangaliev <i(dot)kurbangaliev(at)postgrespro(dot)ru> |
---|---|
To: | Bruce Momjian <bruce(at)momjian(dot)us> |
Cc: | CPT <cpt(at)novozymes(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org> |
Subject: | Re: [GENERAL] Feature Request: bigtsvector |
Date: | 2015-09-09 15:14:28 |
Message-ID: | 20150909181428.2c1da09c@iw |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general pgsql-hackers |
On Wed, 9 Sep 2015 10:52:02 -0400
Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> On Wed, Jun 17, 2015 at 07:58:21AM +0200, CPT wrote:
> > Hi all;
> >
> > We are running a multi-TB bioinformatics system on PostgreSQL and
> > use a denormalized schema in
> > places with a lot of tsvectors aggregated together for centralized
> > searching. This is
> > very important to the performance of the system. These aggregate
> > many documents (sometimes tens of thousands), many of which contain
> > large numbers of references to other documents. It isn't uncommon
> > to have tens of thousands of lexemes. The tsvectors hold mixed
> > document id and natural language search information (all f which
> > comes in from the same documents).
> >
> > Recently we have started hitting the 1MB limit on tsvector size. We
> > have found it possible to
> > patch PostgreSQL to make the tsvector larger but this changes the
> > on-disk layout. How likely is
> > it that either the tsvector size could be increased in future
> > versions to allow for vectors up to toastable size (1GB logical)? I
> > can't imagine we are the only ones with such a problem. Since, I
> > think, changing the on-disk layout might not be such a good idea,
> > maybe it would be worth considering having a new bigtsvector type?
> >
> > Btw, we've been very impressed with the extent that PostgreSQL has
> > tolerated all kinds of loads we have thrown at it.
>
> Can anyone on hackers answer this question from June?
>
Hi, I'm working on patch now that removes this limit without changes (or
small changes) of on-disk layout. I think it'll be ready during this
month.
----
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com <http://www.postgrespro.com/>
The Russian Postgres Company
From | Date | Subject | |
---|---|---|---|
Next Message | Bruce Momjian | 2015-09-09 15:17:18 | Re: [GENERAL] Feature Request: bigtsvector |
Previous Message | Pavel Suderevsky | 2015-09-09 15:06:22 | Re: Buffers: shared hit/read to shared_buffers dependence |
From | Date | Subject | |
---|---|---|---|
Next Message | Bruce Momjian | 2015-09-09 15:17:18 | Re: [GENERAL] Feature Request: bigtsvector |
Previous Message | Andres Freund | 2015-09-09 15:14:01 | Re: Re: [HACKERS] Re: [HACKERS] 答复:[HACKERS] 答复:[HACKERS] about fsync in CLOG buffer write |