Re: Fix for GiST penalty

On 31.05.2011 01:07, Alexander Korotkov wrote:
> In gist_box_penalty function floating point error in line
> *result = (float) (size_box(ud) - size_box(origentry->key));
> sometimes makes *result a very small negative number.
> I beleive that best place to fix it is gistpenalty function. The attached
> patch makes this function treating negative number from user's penalty as
> zero. I didn't find mention of negative penalty value in documentation. So,
> AFAICS such behaviour shouldn't break anything.
> After the patch index performance is ok.

Yeah, there seems to be a hidden assumption that the return value of the
penalty function is always >= 0. The logic in gistchoose() in particular
seems to assume that, by using < 0 to mean uninitialized slots in the
which_grow array.

The documentation should be fixed too.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com