Re: gsoc, oprrest function for text search take 2

From: Jan Urbański <j(dot)urbanski(at)students(dot)mimuw(dot)edu(dot)pl>
To: Heikki Linnakangas <heikki(at)enterprisedb(dot)com>
Cc: Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: gsoc, oprrest function for text search take 2
Date: 2008-08-14 09:53:25
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Heikki Linnakangas wrote:
> Jan Urbański wrote:
>> Not good... Shall I try sorting pg_statistics arrays on text values
>> instead of frequencies?
> Yeah, I'd go with that. If you only do it for the new
> STATISTIC_KIND_MCV_ELEMENT statistics, you shouldn't need to change any
> other code.

OK, will do.

>> BTW: I just noticed some text_to_cstring calls, they came from
>> elog(DEBUG1)s that I have in my code. But they couldn't have skewn the
>> results much, could they?
> Well, text_to_cstring was consuming 1.1% of the CPU time on its own, and
> presumably some of the AllocSetAlloc overhead is attributable to that as
> well. And perhaps some of the detoasting as well.
> Speaking of which, a lot of time seems to be spent on detoasting. I'd
> like to understand that a better. Where is the detoasting coming from?

Hmm, maybe bttext_pattern_cmp does some detoasting? It calls
PG_GETARG_TEXT_PP(), which in turn calls pg_detoast_datum_packed(). Oh,
and also I think that compare_lexeme_textfreq() uses DatumGetTextP() and
that also does detoasting. The root of all evil could by keeping a Datum
in the TextFreq array, and not a "text *", which is something you
pointed out earlier and I apparently didn't understand.

So right now the idea is to:
(1) pre-sort STATISTIC_KIND_MCELEM values
(2) build an array of pointers to detoasted values in tssel()
(3) use binary search when looking for MCELEMs during tsquery analysis


Jan Urbanski
GPG key ID: E583D7D2

ouden estin

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Stehule 2008-08-14 09:56:58 proposal sql: labeled function params
Previous Message Simon Riggs 2008-08-14 09:51:01 Re: [HACKERS] autovacuum: use case for indenpedent TOAST table autovac settings