Re: BUG #5021: ts_parse doesn't recognize email addresses with underscores

Robert Haas escreveu:
> I'm not real familiar with ts_parse(), but I'm thinking that it
> doesn't have any special casing for email addresses and is just
> intended to parse text for full-text-search - in which case splitting
> on _ is a pretty good algorithm.
>
It is a bug. The tsearch claims to identify types of tokens but it doesn't
correctly identify any valid e-mail addresses. As Dan stated ts_parse() fails
to recognize an e-mail address. For example, foo+bar(at)baz(dot)com is a valid e-mail
but the function fails to report that.

It is not that simple to identify an e-mail address that agrees with RFC. As
that code is a state machine, IMHO it decides too early (when it finds _) that
that string is not an e-mail address. AFAIR, that's not an one-line fix.

euler=# select distinct token as email from ts_parse('default',
'foo(dot)bar(at)baz(dot)com');
email
─────────────────
foo(dot)bar(at)baz(dot)com
(1 row)

euler=# select distinct token as email from ts_parse('default',
'foo+bar(at)baz(dot)com');
email
─────────────
foo
+
bar(at)baz(dot)com
(3 rows)

euler=# select distinct token as email from ts_parse('default',
'foo_bar(at)baz(dot)com');
email
─────────────
foo
bar(at)baz(dot)com
_
(3 rows)

--
Euler Taveira de Oliveira
http://www.timbira.com/