Re: parallelize queries containing subplans

On Wed, Dec 28, 2016 at 1:17 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> Currently, queries that have references to SubPlans or
> AlternativeSubPlans are considered parallel-restricted. I think we
> can lift this restriction in many cases especially when SubPlans are
> parallel-safe. To make this work, we need to propagate the
> parallel-safety information from path node to plan node and the same
> could be easily done while creating a plan. Another option could be
> that instead of propagating parallel-safety information from path to
> plan, we can find out from the plan if it is parallel-safe (doesn't
> contain any parallel-aware node) by traversing whole plan tree, but I
> think it is a waste of cycles. Once we have parallel-safety
> information in the plan, we can use that for detection of
> parallel-safe expressions in max_parallel_hazard_walker(). Finally,
> we can pass all the subplans to workers during plan serialization in
> ExecSerializePlan(). This will enable workers to execute subplans
> that are referred in parallel part of the plan. Now, we might be able
> to optimize it such that we pass only subplans that are referred in
> parallel portion of plan, but I am not sure if it is worth the trouble
> because it is one-time cost and much lesser than other things we do
> (like creating
> dsm, launching workers).

It seems unfortunate to have to add a parallel_safe flag to the
finished plan; the whole reason we have the Path-Plan distinction is
so that we can throw away information that won't be needed at
execution time. The parallel_safe flag is, in fact, not needed at
execution time, but just for further planning. Isn't there some way
that we can remember, at the time when a sublink is converted to a
subplan, whether or not the subplan was created from a parallel-safe
path? That seems like it would be cleaner than maintaining this flag
for all plans.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company