Re: [PATCHES] a tsearch2 (8.2.4) dictionary that only filters out stopwords

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>, Jan Urbański <j(dot)urbanski(at)students(dot)mimuw(dot)edu(dot)pl>, pgsql-patches(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org, Teodor Sigaev <teodor(at)sigaev(dot)ru>
Subject: Re: [PATCHES] a tsearch2 (8.2.4) dictionary that only filters out stopwords
Date: 2008-03-07 20:06:24
Message-ID: 200803072006.m27K6OQ24881@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches


Added to TODO:

* Allow text search dictionary to filter out only stop words

http://archives.postgresql.org/pgsql-patches/2007-11/msg00081.php

---------------------------------------------------------------------------

Tom Lane wrote:
> Oleg Bartunov <oleg(at)sai(dot)msu(dot)su> writes:
> > Let's consider one example - removing accents.
> > In the past I always recommend people to use regex functions before
> > to_tsvector conversion to remove accents, but recently I was noticed that
> > such trick doesn't work with headline(). So, the only way is to have
> > special dictionary dict_remove_accent before, which works as a filter.
>
> > I don't remember why do we left this for future releases, though.
>
> That would require a system-to-dictionary API change (to be able to
> modify the token under inspection), no? So it's certainly something
> I'd say is too late for 8.3.
>
> One thought that came to mind is that the option name should be just
> "Accept" not "AcceptAll". To me "All" implies that it would accept
> *everything* ... including stopwords.
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Chernow 2008-03-07 20:09:45 Re: Commitfest process
Previous Message Bruce Momjian 2008-03-07 20:02:00 Re: How to keep a table in memory?

Browse pgsql-patches by date

  From Date Subject
Next Message Bruce Momjian 2008-03-07 20:22:20 Re: Cleaner API for appendStringInfoVA
Previous Message Peter Eisentraut 2008-03-07 18:29:14 Re: Minimum selectivity estimate for LIKE 'prefix%'