Re: GIN index fast list search may become un-interruptible for long time.

Hi Kirill,

Thanks for posting this patch. We have seen this happen in production
a few times this year.

This still reproduces on current master. With your test case and 900k
keys, the query did not respond to pg_cancel_backend for more than 3
minutes. Stack samples show almost all of the time in
collectMatchesForHeapRow, with the pending list page locked. The
numbers are from a cassert build.

> The reason that the loop takes so long is that we look up every key item
> individually on the pending list. In your example, the key has about
> 900000 items. With such a large number of key items, it would be much
> faster to sort the key items, and do a "merge join" of the key items and
> the item on the page. That's a bigger patch though, and not
> backpatchable. But aside from the interrupt issue, it would make it much
> faster.

I would like to know if you plan to post a v2 along the lines Heikki
suggested. If so, I am happy to review and test it. If not, I can work
on it, unless you prefer otherwise.

Thanks,
Shihao