Re: FTS uses "tsquery" directly in the query

From: xu fei <autofei(at)yahoo(dot)com>
To: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: FTS uses "tsquery" directly in the query
Date: 2010-01-25 15:19:59
Message-ID: 683581.73681.qm@web45406.mail.sp1.yahoo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi, Oleg Bartunov:
First thanks for your quick replay. Could you explain it a little more on "it's general limitation/feature"? I just confuse that to_tsquery('item') function will return a tsquery type which is same as 'item'::tsquery, to my understanding. 
Let me explain what I want:First Step: extract top K tokensI have a table with a column as tsvector type. Some records in this column are too big, which contain hundreds tokens. I just want the top K tokens based on the frequency, for example top 5. I am not sure there is a direct way to get such kind top K tokens. I just read them out in Java and count frequency for each token and sort them.
Second Step: generate queryNow I will use these tokens to construct a query to search other vectors in the same table. I can not directly use to_tsquery() due to two reasons: 1) The default logic operator in to_tsquery() is "&" but what I need it "|". 2) Since the tokens are from tsvector, they are already normalized. If I use to_tsquery() again, they will be normalized again! For example, “course” -> “cours” -> “cour”. So I just concatenate the top K tokens with “|” and directly use "::tsquery ". 
Unfortunately, as you say "it's general limitation/feature”, I can not do that. I checked your manual “Full-Text Search in PostgreSQL A Gentle Introduction”, but could not figure out how. So is it possible to implement what I want in FTS? If so, how?
Thank! 
Xu
--- On Sun, 1/24/10, Oleg Bartunov <oleg(at)sai(dot)msu(dot)su> wrote:

From: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
Subject: Re: [GENERAL] FTS uses "tsquery" directly in the query
To: "xu fei" <autofei(at)yahoo(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Date: Sunday, January 24, 2010, 2:11 AM

Xu,

FTS has nothing with your problem, it's general limitation/feature.

Oleg
On Sat, 23 Jan 2010, xu fei wrote:

> Hi, everyone:
> First I can successful run this query:select name, ts_rank_cd(vectors, query) as rank from element, to_tsquery('item') query where query @@ vectors order by rank desc;But actually I want to run this one:select name, ts_rank_cd(vectors, query) as rank from element, 'item'::tsquery query where query @@ vectors order by rank desc;Looks like that FTS does not support directly use "::tsquery " in such query. Do I misunderstand something?  Thanks!
> Xu
>
>
>

    Regards,
        Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message A. Kretschmer 2010-01-25 16:07:45 Re: Log full of: statement_timeout out of the valid range.
Previous Message Abraham, Danny 2010-01-25 15:09:13 Log full of: statement_timeout out of the valid range.