Re: WIP: Fast GiST index build

On 27.06.2011 13:45, Alexander Korotkov wrote:
> I've added information about testing on some real-life dataset to wiki page.
> This dataset have a speciality: data is ordered inside it. In this case
> tradeoff was inverse in comparison with expectations about "fast build"
> algrorithm. Index built is longer but index quality is significantly better.
> I think high speed of regular index built is because sequential inserts are
> into near tree parts. That's why number of actual page reads and writes is
> low. The difference in tree quality I can't *convincingly explain now.*
> I've also maked tests with shuffled data of this dataset. In this case
> results was similar to random generated data.

Once again, interesting results.

The penalty function is called whenever a tuple is routed to the next
level down, and the final tree has the same depth with and without the
patch, so I would expect the number of penalty calls to be roughly the
same. But clearly there's something wrong with that logic; can you
explain in layman's terms why the patch adds so many gist penalty calls?
And how many calls does it actually add, can you gather some numbers on
that? Any ides on how to mitigate that, or do we just have to live with
it? Or maybe use some heuristic to use the existing insertion method
when the patch is not expected to be helpful?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com