Re: WIP: Fast GiST index build

On 22.08.2011 13:23, Alexander Korotkov wrote:
> On Wed, Aug 17, 2011 at 11:11 AM, Alexander Korotkov
> <aekorotkov(at)gmail(dot)com>wrote:
>
>> On Tue, Aug 16, 2011 at 11:15 PM, Heikki Linnakangas<
>> heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>>
>>> On 16.08.2011 22:10, Heikki Linnakangas wrote:
>>>
>>>> Here's an version of the patch with a bunch of minor changes:
>>>>
>>>
>>> And here it really is, this time with an attachment...
>>
>> Thanks a lot. I'm going to start rerunning the tests now.
>>
>
> First bunch of test results will be available soon (tests running and
> results processing take some time). While there is a patch with few small
> bugfixes.

I've been mulling this through, and will continue working on this
tomorrow, but wanted to share this version meanwhile:

* Moved all the buffering build logic from gistplacetopage() to a new
function in gistbuild.c. There's almost no changes to gistplacetopage()
now, it returns the SplitInfo struct as usual, and the new function
deals with that and handles the call to
gistRelocateBuildBuffersOnSplit(), and the recursion to insert downlinks.

* Simplified the handling of buffersOnLevels lists a bit. There's now an
entry in buffersOnLevels array for all levels, even those that don't
have buffers because levelStep > 1. That wastes a few bytes in the
array, but it's more easy to debug and understand that way. Also,
there's no separate Len and Count variables for it anymore.

* Moved validateBufferingOption() to gistbuild.c

* Moved the code to add buffer to emptying queue to
gistPushItupToNodeBuffer() (was handled by the callers previously)

* Removed gistGetNodeBufferBusySize(), it was unused

* A lot of comment changes

Could you share the test scripts, patches and data sets etc. needed to
reproduce the tests you've been running? I'd like to try them out on a
test server.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com