pgsql: Make the upper part of the planner work by generating and compar

Make the upper part of the planner work by generating and comparing Paths.

I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.

In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)

There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.

Thanks to Robert Haas, David Rowley, and Amit Kapila for review.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/3fc6e2d7f5b652b417fa6937c34de2438d60fa9f

Modified Files
--------------
doc/src/sgml/fdwhandler.sgml | 34 +
src/backend/executor/execAmi.c | 23 +-
src/backend/nodes/copyfuncs.c | 2 +-
src/backend/nodes/outfuncs.c | 238 ++-
src/backend/nodes/readfuncs.c | 4 +-
src/backend/optimizer/README | 108 +-
src/backend/optimizer/path/allpaths.c | 133 +-
src/backend/optimizer/path/costsize.c | 152 +-
src/backend/optimizer/path/equivclass.c | 42 -
src/backend/optimizer/path/pathkeys.c | 9 +-
src/backend/optimizer/plan/createplan.c | 1900 ++++++++++++++-----
src/backend/optimizer/plan/planagg.c | 320 +---
src/backend/optimizer/plan/planmain.c | 7 +-
src/backend/optimizer/plan/planner.c | 2856 ++++++++++++-----------------
src/backend/optimizer/plan/setrefs.c | 72 +-
src/backend/optimizer/plan/subselect.c | 180 +-
src/backend/optimizer/prep/prepjointree.c | 7 +-
src/backend/optimizer/prep/prepunion.c | 696 ++++---
src/backend/optimizer/util/pathnode.c | 1043 ++++++++++-
src/backend/optimizer/util/plancat.c | 22 +-
src/backend/optimizer/util/relnode.c | 59 +-
src/backend/optimizer/util/tlist.c | 154 +-
src/include/nodes/nodes.h | 54 +-
src/include/nodes/plannodes.h | 32 +-
src/include/nodes/relation.h | 295 ++-
src/include/optimizer/cost.h | 7 +-
src/include/optimizer/pathnode.h | 98 +-
src/include/optimizer/paths.h | 8 +-
src/include/optimizer/planmain.h | 45 +-
src/include/optimizer/planner.h | 11 +-
src/include/optimizer/prep.h | 3 +-
src/include/optimizer/subselect.h | 8 +-
src/include/optimizer/tlist.h | 11 +-
src/test/regress/expected/aggregates.out | 9 +-
src/test/regress/expected/join.out | 39 +-
35 files changed, 5573 insertions(+), 3108 deletions(-)