Re: Remove 1MB size limit in tsvector

On Thu, 10 Aug 2017 18:06:17 +0300
Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru> wrote:

> On Wed, Aug 9, 2017 at 7:38 PM, Robert Haas <robertmhaas(at)gmail(dot)com>
> wrote:
>
> > On Tue, Aug 1, 2017 at 4:00 PM, Ildus K
> > <i(dot)kurbangaliev(at)postgrespro(dot)ru> wrote:
> > > It's a workaround. DatumGetTSVector and
> > > DatumGetTSVectorCopy will upgrade tsvector on the fly if it
> > > has old format.
> >
> > Hmm, that seems like a real fix, not just a workaround. If you can
> > transparently read the old format, there's no problem. Not sure
> > about performance, though.
> >
>
> +1
> Ildus, I think we need to benchmark reading of the old format. There
> would be tradeoff between performance of old format reading and
> amount of extra code needed. Once we will have benchmarks we can
> consider whether this is the solution we would like to buy.

In my benchmarks when database fits into buffers (so it's measurement of
the time required for the tsvectors conversion) it gives me these
results:

Without conversion:

$ ./tsbench2 -database test1 -bench_time 300
2017/08/17 12:04:44 Number of connections: 4
2017/08/17 12:04:44 Database: test1
2017/08/17 12:09:44 Processed: 51419

With conversion:

$ ./tsbench2 -database test1 -bench_time 300
2017/08/17 12:14:31 Number of connections: 4
2017/08/17 12:14:31 Database: test1
2017/08/17 12:19:31 Processed: 43607

I ran a bunch of these tests, and these results are stable on my
machine. So in these specific tests performance regression about 15%.

Same time I think this could be the worst case, because usually data
is on disk and conversion will not affect so much to performance.

--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company