Re: Parallel Seq Scan

On Mon, Feb 9, 2015 at 7:37 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Mon, Feb 9, 2015 at 2:31 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
wrote:
> > Another idea is to use Executor level interfaces (like ExecutorStart(),
> > ExecutorRun(), ExecutorEnd()) for execution rather than using Portal
> > level interfaces. I have used Portal level interfaces with the
> > thought that we can reuse the existing infrastructure of Portal to
> > make parallel execution of scrollable cursors, but as per my analysis
> > it is not so easy to support them especially backward scan, absolute/
> > relative fetch, etc, so Executor level interfaces seems more appealing
> > to me (something like how Explain statement works (ExplainOnePlan)).
> > Using Executor level interfaces will have advantage that we can reuse
them
> > for other parallel functionalaties. In this approach, we need to take
> > care of constructing relavant structures (with the information passed by
> > master backend) required for Executor interfaces, but I think these
should
> > be lesser than what we need in previous approach (extract seqscan
specific
> > stuff from executor).
>
> I think using the executor-level interfaces instead of the
> portal-level interfaces is a good idea. That would possibly let us
> altogether prohibit access to the portal layer from within a parallel
> worker, which seems like it might be a good sanity check to add. But
> that seems to still require us to have a PlannedStmt and a QueryDesc,
> and I'm not sure whether that's going to be too much of a pain. We
> might need to think about an alternative API for starting the Executor
> like ExecutorStartParallel() or ExecutorStartExtended(). But I'm not
> sure. If you can revise things to go through the executor interfaces
> I think that would be a good start, and then perhaps after that we can
> see what else makes sense to do.
>

Okay, I have modified the patch to use Executor level interfaces
rather than Portal-level interfaces. To achieve that I need to add
a new Dest (DestRemoteBackend). For now, I have modified
printtup.c to handle this new destination type similar to what
it does for DestRemote and DestRemoteExecute.

Apart from above, the other major changes to address your concerns
and review comments are:
a. Made InitiateWorkers() and ParallelQueryMain(an entry function for
parallel query execution) modular
b. Adapted the parallel-heap-scan patch posted by Robert upthread
http://www.postgresql.org/message-id/CA+TgmoYJETgeAXUsZROnA7BdtWzPtqExPJNTV1GKcaVMgSdhug@mail.gmail.com
c. Now master and worker backend, both run as part of same node
ParallelSeqScan (I have yet to update copy and out funcs for new
parameters), check if you think that is the right way to go. I still
feel it would have been better if master and backend worker runs
as part of different nodes, however this also looks okay for the
purpose of parallel sequential scan.

I have yet to modify the code to allow expressions in projection
and allowing joins, I think these are related to allow-parallel-safety
patch, I will once take a look at that patch and then modify
accordingly.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com