Re: [PATCHES] GIN improvements

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Cc: Teodor Sigaev <teodor(at)sigaev(dot)ru>, Pgsql Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCHES] GIN improvements
Date: 2008-07-23 21:26:49
Message-ID: 13239.1216848409@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> Tom Lane wrote:
>> It's a mess:

> These are rather severe problems. Maybe there's a better solution, but
> perhaps it would be good enough to lock out concurrent access to the
> index while the bulkinsert procedure is working.

Ugh...

The idea I was toying with was to not allow GIN scans to "stop" on
pending-insertion pages; rather, they should suck out all the matching
tuple IDs into backend-local memory as fast as they can, and then return
the TIDs to the caller one at a time from that internal array. Then,
when the scan is later visiting the main part of the index, it could
check each matching TID against that array to see if it'd already
returned the TID. (So it might be an idea to sort the TID array after
gathering it, to make those subsequent checks fast via binary search.)

This would cost in backend-local memory, of course, but hopefully not
very much. The advantages are the elimination of the deadlock risk
from scan-blocks-insertcleanup-blocks-insert, and fixing the race
condition when a TID previously seen in the pending list is moved to
the main index. There were still a number of locking issues to fix
but I think they're all relatively easy to deal with.

regards, tom lane

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Dann Corbit 2008-07-23 22:50:38 Re: Research/Implementation of Nested Loop Join optimization
Previous Message Dimitri Fontaine 2008-07-23 21:08:19 PostgreSQL extensions packaging

Browse pgsql-patches by date

  From Date Subject
Next Message Simon Riggs 2008-07-23 23:05:56 Re: [PATCHES] odd output in restore mode
Previous Message Andrew Dunstan 2008-07-23 21:01:18 Re: [PATCHES] odd output in restore mode