Re: Yet another fast GiST build

On 09/09/2020 13:28, Andrey M. Borodin wrote:
> Thanks Darafei!
>
>> 9 сент. 2020 г., в 12:05, Darafei Komяpa Praliaskouski
>> <me(at)komzpa(dot)net> написал(а):
>>
>>> How does the 'sortsupport' routine interact with
>>> 'compress'/'decompress'? Which representation is passed to the
>>> comparator routine: the original value from the table, the
>>> compressed representation, or the decompressed representation? Do
>>> the comparetup_index_btree() and readtup_index() routines agree
>>> with that?
>>
>> Currently we pass compressed values, which seems not very good. But
>> there was a request from PostGIS maintainers to pass values before
>> decompression. Darafei, please, correct me if I'm wrong. Also can
>> you please provide link on PostGIS B-tree sorting functions?
>>
>> We were expecting to reuse btree opclass for this thing. This way
>> btree_gist extension will become a lot thinner. :)
>
> I think if we aim at reusing B-tree sort support functions we have to
> pass uncompressed values. They can be a lot bigger and slower in case
> of PostGIS. We will be sorting actual geometries instead of MBRs.
>
> In my view it's better to implement GiST-specific sort support in
> btree_gist, rather than trying to reuse existing sort supports.

Yeah, I don't think reusing existing sortsupport functions directly is
important. The comparison function should be short anyway for
performance reasons, so it won't be a lot of code to copy-paste. And if
there are some common subroutines, you can put them in a separate
internal functions for reuse.

Using the 'compressed' format seems reasonable to me. It's natural to
the gistbuild.c code, and the comparison routine can 'decompress' itself
if it wishes. If the decompressions is somewhat expensive, it's
unfortunate if you need to do it repeatedly in the comparator, but
tuplesort.c would need pretty big changes to keep around a separate
in-memory representation compare. However, you could use the sort
"abbreviation" functionality to mitigate that.

Come to think of it, the point z-order comparator could benefit a lot
from key abbreviation, too. You could do the point -> zorder conversion
in the abbreviation routine.

- Heikki