Re: english parser in text search: support for multiple words in the same position

Hi,

On 08/02/2010 03:12 PM, Sushant Sinha wrote:
> The current text parser already returns url and url_path. That already
> increases the number of unique tokens.

Well, I think I simply turned that off to be able to search for plain
words. It still works for complete URLs, those are just treated like
text, then.

> Earlier people have expressed the need to index urls/emails and
> currently the text parser already does so. Reverting that would be a
> regression of functionality. Further, a ranking function can take
> advantage of direct match of a token.

That's a point, yes. However, simply making the same string turn up
twice in the tokenizer's output doesn't sound like the right solution to
me. Especially considering that the query parser uses the very same
tokenizer.

Regards

Markus Wanner