Re: a tsearch2 (8.2.4) dictionary that only filters out stopwords

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Jan Urbański <j(dot)urbanski(at)students(dot)mimuw(dot)edu(dot)pl>
Cc: pgsql-patches(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org, Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>, Teodor Sigaev <teodor(at)sigaev(dot)ru>
Subject: Re: a tsearch2 (8.2.4) dictionary that only filters out stopwords
Date: 2007-11-14 16:29:07
Message-ID: 24761.1195057747@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

This patch:
http://archives.postgresql.org/pgsql-patches/2007-11/msg00137.php
seems simple and useful enough that I think we ought to slip it into
8.3, even though we are far past feature freeze.

As the "simple" dictionary type stands in CVS HEAD, it is only useful as
the last dictionary in a stack, since it never passes anything on as
unrecognized. With the proposed AcceptAll = false option, it could be
used to filter out some stopwords before feeding tokens to another
dictionary. While most dictionary types have their own stopword support,
some of them match stopwords after their own normalization processing,
and so there's no way to filter on pre-normalized words. That seems
like a good improvement, even without the specific need-example that
Jan provided at the start of the thread.

Normally we'd never consider adding a new feature so late in the
development cycle, but this seems small enough and useful enough
to make an exception. Comments?

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2007-11-14 16:42:19 Re: a tsearch2 (8.2.4) dictionary that only filters out stopwords
Previous Message Tom Lane 2007-11-14 15:54:03 Re: Fix pg_dump dependency on postgres.h

Browse pgsql-patches by date

  From Date Subject
Next Message Bruce Momjian 2007-11-14 16:42:19 Re: a tsearch2 (8.2.4) dictionary that only filters out stopwords
Previous Message Tom Lane 2007-11-14 15:54:03 Re: Fix pg_dump dependency on postgres.h