Re: GIN index fast list search may become un-interruptible for long time.

From: shihao zhong <zhong950419(at)gmail(dot)com>
To: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Cc: Kirill Reshke <reshkekirill(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: GIN index fast list search may become un-interruptible for long time.
Date: 2026-09-25 22:03:16
Message-ID: CAGRkXqTAf2ghC9hyc_NLx5oA0PP=v3L8S8LMQRcDxbr9GJ6niQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Heikki,

> With such a large number of key items, it would be much
> faster to sort the key items, and do a "merge join" of the key items and
> the item on the page.

Here is a patch set along those lines, v2 attached. The locking is
unchanged.

0001 sorts the query entries once. For each pending list row, it then
looks up the row's keys in them with a binary search. That is faster
than a plain merge, because a row usually has far fewer keys than the
query. In Kirill's case, a 1s statement_timeout fires after 202s on
master and after 1s with 0001. With a default size pending list, 1000
entry queries are 22x faster. Queries with fewer than 64 entries take
the old path.

0002 calls the consistent functions after the pending list lock is
released. It helps when many rows match. 0001 can go in without it.

0003 adds tests.

Large arrays are still slow in startScanKey(), but it holds no lock.

Thanks,
Shihao

Attachment Content-Type Size
v2-0002-Call-GIN-consistent-functions-for-pending-list-ro.patch application/x-patch 7.3 KB
v2-0001-Speed-up-GIN-pending-list-scans-for-scan-keys-wit.patch application/x-patch 11.8 KB
v2-0003-Add-tests-for-GIN-pending-list-scans-with-many-sc.patch application/x-patch 6.5 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2026-09-25 22:23:25 Re: Fix "unexpected logical decoding status change" error; from concurrent logical decoding activation
Previous Message Melanie Plageman 2026-09-25 21:54:01 Hardening visibility-map maintenance and recovery