Re: BUG #16388: Different results when bitmap scan enabled/disabled

PG Bug reporting form <noreply(at)postgresql(dot)org> writes:
> When Postgres uses a bitmap heap scan to evaluate a tsquery that includes !,
> it is giving me different (and incorrect) results compared to when it
> performs a seqscan.
> Can anybody shed some light on this?

It's a bug, without a doubt.

It looks to me like what is happening is that when gin_tsquery_consistent
asks TS_execute if the query is matched, we recurse down to
TS_phrase_execute which does this:

/*
* If either operand has no position information, then we can't
* return position data, only a "possible match" result. "Possible
* match" answers are only wanted when TS_EXEC_PHRASE_NO_POS flag
* is set, otherwise return false.
*/
if ((Ldata.npos == 0 && !Ldata.negate) ||
(Rdata.npos == 0 && !Rdata.negate))
return (flags & TS_EXEC_PHRASE_NO_POS) ? true : false;

so that returns "true" up to the calling TS_execute level:

case OP_NOT:
if (flags & TS_EXEC_CALC_NOT)
return !TS_execute(curitem + 1, arg, flags, chkcond);
else
return true;

which returns "false" and we conclude the index entry doesn't match.

AFAICS this is fundamentally broken and the only way to un-break it
is to introduce explicit three-valued logic, ie we have to return
"maybe" which OP_NOT mustn't invert. Various people have tried to
wiggle around that by inventing assorted more-or-less-bogus flags,
like the TS_EXEC_PHRASE_NO_POS flag seen above, but in the end it
Just Does Not Work to take shortcuts.

Depressingly, although tsginidx.c has decided to invent its
own version with three-valued logic (TS_execute_ternary),
that isn't being used in this scenario ... and it looks like
it'd get the case wrong if it were used, because it's got its
own set of bogosities.

I'll see about getting this fixed in time for next month's
minor releases, but it's definitely not a trivial change.

Thanks for the report!

regards, tom lane