Re: BUG #19748: tsvectorrecv accepts duplicate lexemes and position 0 that tsvectorin rejects

PG Bug reporting form <noreply(at)postgresql(dot)org> writes:
> tsvectorrecv() (src/backend/utils/adt/tsvector.c:446) has this comment
> (tsvector.c:552-556):

> * Enforce that datalen is still within MAXSTRPOS, ie the last lexeme
> * didn't go past that. We could allow that, since no "pos" field
> * overflowed, but tsvectorrecv shouldn't accept values that other
> * tsvector-constructing routines wouldn't.

> Two checks are still missing relative to tsvectorin():

> 1. Duplicate lexemes are not merged. tsvectorin() runs uniqueentry()
> (tsvector.c:265). tsvectorrecv() detects "compareentry(...) <= 0"
> (tsvector.c:514-517), but only sorts (tsvector.c:563); equal entries
> stay. The result is a tsvector with repeated lexemes, which no other
> constructor produces, and which silently changes on a text round trip.

Hmph. Making that happen would be a lot of mess, since that'd require
also merging and de-duping positions for equal lexemes, and the de-dup
logic used by tsvectorin() can't be re-used because it doesn't work on
the same data representation. But it seems to me that tsvectorrecv()
is being quite schizophrenic here: it will let you be sloppy about
lexeme order, but not about duplicated or out-of-order positions.
I don't see a good reason for it to be forgiving about either thing.
So my proposal is to just throw an error if the lexeme sequence isn't
strictly increasing, as it already does for positions. That is
probably not something to back-patch, but this issue is surely not
important enough to worry about back-patching.

> 2. The first position is never checked. tsvectorin() rejects position 0
> ("wrong position info in tsvector", tsvector_parser.c:334); the receive
> loop only checks that positions ascend for j > 0 (tsvector.c:540-545),
> so a leading position 0 is accepted.

Agreed, that's an oversight.

regards, tom lane