Re: WIP: Fast GiST index build

On 12.07.2011 11:34, Alexander Korotkov wrote:
> New version of patch with a little more refactoring and comments.

Great! The README helps tremendously to understand this, thanks for that.

One thing that caught my eye is that when you empty a buffer, you load
the entire subtree below that buffer, down to the next buffered or leaf
level, into memory. Every page in that subtree is kept pinned. That is a
problem; in the general case, the buffer manager can only hold a modest
number of pages pinned at a time. Consider that the minimum value for
shared_buffers is just 16. That's unrealistically low for any real
system, but the default is only 32MB, which equals to just 4096 buffers.
A subtree could easily be larger than that.

I don't think you're benefiting at all from the buffering that BufFile
does for you, since you're reading/writing a full block at a time
anyway. You might as well use the file API in fd.c directly, ie.
OpenTemporaryFile/FileRead/FileWrite.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com