Skip site navigation (1) Skip section navigation (2)

Re: [PATCHES] GIN improvements

From: Teodor Sigaev <teodor(at)sigaev(dot)ru>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Pgsql Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCHES] GIN improvements
Date: 2008-10-31 13:27:22
Message-ID: 490B07BA.6030109@sigaev.ru (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
Reworked version of fast insertion patch for GIN.

>> * shiftList does LockBufferForCleanup, which means that it can be blocked
>> for an indefinitely long time by a concurrent scan, and since it's 
>> holding
>> exclusive lock on metapage no new scans or insertions can start 
>> meanwhile.
>> This is not only horrid from a performance standpoint but it very 
>> probably
>> can result in deadlocks --- which will be deadlocks on LWLocks and thus
>> not even detected by the system.
> Ops, I see possible scenario: UPDATE tbl SET gin_indexed_field = ... 
> where gin_indexed_field ....  with concurrent shiftList. Will fix. Thank 
> you.
Fixed, see below

>> * GIN index scans release lock and pin on one pending-list page before
>> acquiring pin and lock on the next, which means there's a race condition:
>> shiftList could visit and delete the next page before we get to it,
>> because there's a window where we're holding no buffer lock at all.
> Agree, will fix.
Fixed

>> * There is a bigger race condition, which is that after a scan has
>> returned a tuple from a pending page, vacuum could move the index entry
>> into the main index structure, and then that same scan could return that
>> same index entry a second time.  This is a no-no, and I don't see any 
>> easy
>> fix.
Fixed. TIDBitmap is used for that and for preventing deadlock mentioned above.
TIDBitmap is used for collectiong matched tuples from pending pages and after 
that it used as filter for results from regular GIN's scan.

Patch extends TIDBitmap interface by 2 functions:
bool tbm_check_tuple(TIDBitmap *tbm, const ItemPointer tid);
	returns true if tid already exists in bitmap
bool   tbm_has_lossy(TIDBitmap *tbm);
	returns true if bitmap becomes lossy


Also, sequential scan on pending page is replaced to binary search for 
performance. It's not a big win but it might improve performance.
-- 
Teodor Sigaev                                   E-mail: teodor(at)sigaev(dot)ru
                                                    WWW: http://www.sigaev.ru/

Attachment: fast_insert_gin-0.15.gz
Description: application/x-tar (17.6 KB)

In response to

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2008-10-31 13:38:02
Subject: Re: WIP patch: convert SQL-language functions to return tuplestores
Previous:From: Heikki LinnakangasDate: 2008-10-31 13:15:35
Subject: Re: Synchronous replication patch v1

pgsql-patches by date

Next:From: Nikhil SontakkeDate: 2008-10-31 14:50:46
Subject: Re: Fwd: [PATCHES] Auto Partitioning Patch - WIP version 1
Previous:From: Nikhil SontakkeDate: 2008-10-23 12:41:41
Subject: Re: Fwd: [PATCHES] Auto Partitioning Patch - WIP version 1

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group