Re: Should HashSetOp go away

I wrote:
> * create_setop_path's required-space estimate of entrysize * numGroups
> was rather lame before commit 5dfc19814, and it's even more so
> afterwards. It's basically only accounting for the tuples themselves,
> and not either the hashtable overhead or the SetOpStatePerGroupData
> counter space. With wide tuples that might disappear into the noise,
> but with only 16-ish data bytes per tuple it's all about the overhead.
> On my machine this example uses 80 bytes per tuple in the "SetOp hash
> table" context and another 16 or more in the simplehash hashtable.
> So about triple what the planner thought.

> * To do better, we probably need to take this computation out of the
> planner and have execGrouping.c expose a function to estimate the
> TupleHashTable size for N entries and such-and-such average data
> width. That in turn will require simplehash.h to expose a function
> for estimating the size of its tables, because AFAICS its callers are
> not supposed to know such details.

Here's a pair of patches to try to do better. The first one
is concerned with getting more realistic size estimates for
TupleHashTables in the planner. The second is some mop-up
that's been pending for a long time in the same area, namely
getting rid of "long int" field types in Plan nodes.

With 0001, the planner's estimate of the amount of space needed
for your example query seems to be pretty dead-on, at least in
non-debug builds.

regards, tom lane