Re: Empty string in lexeme for tsvector

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Jean-Christophe Arnu <jcarnu(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Empty string in lexeme for tsvector
Date: 2021-11-03 20:40:57
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Jean-Christophe Arnu <jcarnu(at)gmail(dot)com> writes:
>> (By the same token, I think there's a good argument for
>> tsvector_delete_arr to just ignore nulls, not throw an error.
>> That's a somewhat orthogonal issue, though.)

> Nulls are now ignored in tsvector_delete_arr().

I think setweight() with an array should handle this the same as
ts_delete() with an array, so the attached v6 does it like that.

> I also wonder if we should not also consider changing COPY FROM behaviour
> on empty string lexemes.
> Current version is just crashing on empty string lexemes. Should
> we allow them anyway as COPY FROM input (it seems weird not to be able
> to re-import dumped data) or "skipping them" just like array_to_tsvector()
> does in the patched version (that may lead to database content changes) or
> finally should we not change COPY behaviour ?

No, I don't think so. tsvector's restriction against empty lexemes was
certainly intentional from the beginning, so I wouldn't be surprised if
we'd run into semantic difficulties if we remove it. Moreover, we're
going on fourteen years with that restriction and we've had few
complaints, so there's no field demand to loosen it. It's clearly just
an oversight that array_to_tsvector() failed to enforce the restriction.

I polished the docs and tests a bit more, too. I think the attached
is committable -- any objections?

regards, tom lane

Attachment Content-Type Size
empty_string_in_tsvector_v6.patch text/x-diff 9.5 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2021-11-03 20:43:07 Re: On login trigger: take three
Previous Message Daniel Gustafsson 2021-11-03 20:10:25 Re: remove internal support in pgcrypto?