Re: Group-count estimation statistics

> From: Sailesh Krishnamurthy <sailesh(at)cs(dot)berkeley(dot)edu>
> >>>>> "Tom" == Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:
>
> Tom> The only real solution, of course, is to acquire cross-column
> Tom> statistics, but I don't see that happening in the near
> Tom> future.
>
> Another approach is a hybrid hashing scheme where we use a hash table
> until we run out of memory at which time we start spilling to disk. In
> other words, no longer use SortAgg at all ..
>
> Under what circumstances will a SortAgg consumer more IOs than a
> hybrid hash strategy ?

Goetz Graefe did a heck of a lot of analysis of this, prior to his being snapped
up by Microsoft. He also worked out a lot of the nitty-gritty for hybrid hash
algorithms, extending the Grace hash for spill-to-disk, and adding a kind of
recursion for really huge sets. The figures say that hybrid hash beats
sort-aggregate, across the board.