Re: Remove 1MB size limit in tsvector

From: Ildus K <i(dot)kurbangaliev(at)postgrespro(dot)ru>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Remove 1MB size limit in tsvector
Date: 2017-08-01 19:10:22
Message-ID: 20170801221022.107825c6@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, 1 Aug 2017 14:56:54 -0400
Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Tue, Aug 1, 2017 at 10:08 AM, Ildus Kurbangaliev
> <i(dot)kurbangaliev(at)postgrespro(dot)ru> wrote:
> > Historically tsvector type can't hold more than 1MB data.
> > I want to propose a patch that removes that limit.
> >
> > That limit is created by 'pos' field from WordEntry, which have only
> > 20 bits for storage.
> >
> > In the proposed patch I removed this field and instead of it I keep
> > offsets only at each Nth item in WordEntry's array.
>
> So this would break pg_upgrade for tsvector columns?
>

I added a function that will convert old tsvectors on the fly. It's the
approach used in hstore before.

Regards,
Ildus Kurbangaliev

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Erik Rijkers 2017-08-01 19:15:14 Re: parallel documentation improvements
Previous Message Robert Haas 2017-08-01 19:07:38 Re: Update comments in nodeModifyTable.c