Re: parallel append vs. simple UNION ALL

On Sat, Dec 23, 2017 at 4:53 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> As I mentioned in the commit message for the Parallel Append commit
> (ab72716778128fb63d54ac256adf7fe6820a1185), it's kind of sad that this
> doesn't work with UNION ALL queries, which are an obvious candidate
> for such parallelization. It turns out that it actually does work to
> a limited degree: assuming that the UNION ALL query can be converted
> to a simple appendrel, it can consider a parallel append of
> non-partial paths only. The attached patch lets it consider a
> parallel append of partial paths ...

Here's an extended series of patches that now handles both the simple
UNION ALL case (where we flatten it) and the unflattened case:

0001 is pretty much the same as the subquery-smarts.patch file I
attached to the previous email. I don't see much reason not to go
ahead and commit this, although it could use a test case. It makes
the simple/flattened case work. After some study I think that the
gather-parameter handling is correct, although if somebody felt like
reviewing that portion especially I wouldn't say no.

0002 rewrites recurse_union_children to work iteratively rather than
recursively and renames it to plan_union_children. This probably
isn't 100% necessary, but it seems to me that the resulting code is
easier to understand, and it reduces the risk of blowing out the
stack. There should be no user-visible behavior change.

0003 rewrites the setop planner to create a separate upper rel for
each stage of setop planning and uses them to return paths instead of
returning paths directly. This is necessary preparatory work for
anything that wants to consider multiple possible paths for queries
that go through the full setop planner, but it shouldn't have any
visible impact all by itself.

0004 causes generate_union_path() to consider both the traditional
method and also Gather -> Parallel Append -> [partial path for each
subquery]. This is still a bit rough around the edges and there's a
lot more that could be done here, but I'm posting what I have for now
in the (perhaps vain) hope of getting some feedback. With this, you
can use Parallel Append for the UNION ALL step of a query like SELECT
.. UNION ALL .. SELECT ... EXCEPT SELECT ...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company