pgsql: Make the overflow guards in ExecChooseHashTableSize be more

Log Message:
-----------
Make the overflow guards in ExecChooseHashTableSize be more protective.
The original coding ensured nbuckets and nbatch didn't exceed INT_MAX,
which while not insane on its own terms did nothing to protect subsequent
code like "palloc(nbatch * sizeof(BufFile *))". Since enormous join size
estimates might well be planner error rather than reality, it seems best
to constrain the initial sizes to be not more than work_mem/sizeof(pointer),
thus ensuring the allocated arrays don't exceed work_mem. We will allow
nbatch to get bigger than that during subsequent ExecHashIncreaseNumBatches
calls, but we should still guard against integer overflow in those palloc
requests. Per bug #5145 from Bernt Marius Johnsen.

Although the given test case only seems to fail back to 8.2, previous
releases have variants of this issue, so patch all supported branches.

Tags:
----
REL8_3_STABLE

Modified Files:
--------------
pgsql/src/backend/executor:
nodeHash.c (r1.116 -> r1.116.2.1)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/executor/nodeHash.c?r1=1.116&r2=1.116.2.1)