Re: GIN, partial matches, lossy bitmaps

On Mon, Mar 2, 2009 at 2:14 PM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> While reading the GIN code, I just rediscovered that the GIN partial match
> support suffers from the same problem that I criticized the "fast insert"
> patch about, and searching the archives I found that I already complained
> about that back in April:
>
> http://archives.postgresql.org/pgsql-patches/2008-04/msg00157.php
>
> If I'm reading the code correctly, item pointers of all matching heap tuples
> are first collected into a TIDBitmap, and then amgetnext returns tuples from
> that one by one. If the bitmap becomes lossy, an error is thrown.
> gingetbitmap is a dummy implementation: it creates a new TIDBitmap and
> inserts all the tuples from the other TIDBitmap into it one by one, and then
> returns the new TIDBitmap.

The latest version of the path no longer does this - instead, it
flushes the pending list to the main index if the bitmap becomes
lossy. That strikes me as more tolerable than throwing an error, but
I agree with your criticism: I'm not sure why we are insisting on
using a TIDBitmap (which is designed to be lossy at times) in a
situation where we actually can't tolerate lossiness. In fact, this
was the main point of my original review of this patch.

http://archives.postgresql.org/message-id/603c8f070902101859j91fb78eg7e0228afe8f2fd36@mail.gmail.com

> If we remove the support for regular, non-bitmap, index scans with GIN, that
> could be cleaned up as well. Even if we don't do that, gingetbitmap should
> not error when the bitmap becomes lossy, but just return the lossy bitmap.

Make sense to me.

...Robert