Re: Empty string in lexeme for tsvector

Jean-Christophe Arnu <jcarnu(at)gmail(dot)com> writes:
>> (By the same token, I think there's a good argument for
>> tsvector_delete_arr to just ignore nulls, not throw an error.
>> That's a somewhat orthogonal issue, though.)

> Nulls are now ignored in tsvector_delete_arr().

I think setweight() with an array should handle this the same as
ts_delete() with an array, so the attached v6 does it like that.

> I also wonder if we should not also consider changing COPY FROM behaviour
> on empty string lexemes.
> Current version is just crashing on empty string lexemes. Should
> we allow them anyway as COPY FROM input (it seems weird not to be able
> to re-import dumped data) or "skipping them" just like array_to_tsvector()
> does in the patched version (that may lead to database content changes) or
> finally should we not change COPY behaviour ?

No, I don't think so. tsvector's restriction against empty lexemes was
certainly intentional from the beginning, so I wouldn't be surprised if
we'd run into semantic difficulties if we remove it. Moreover, we're
going on fourteen years with that restriction and we've had few
complaints, so there's no field demand to loosen it. It's clearly just
an oversight that array_to_tsvector() failed to enforce the restriction.

I polished the docs and tests a bit more, too. I think the attached
is committable -- any objections?

regards, tom lane