Re: Support Parallel Query Execution in Executor

On Sun, 2006-04-09 at 17:11 +0800, Qingqing Zhou wrote:
> "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote
> >
> > This is exactly the bit of optimism I was questioning. We've already
> > been sweating blood trying to reduce multiprocessor contention on data
> > structures in which collisions ought to be avoidable (ie, buffer arrays
> > where you hope not everyone is hitting the same buffer at once). I
> > think passing large volumes of data between different processes is going
> > to incur quite a lot of locking overhead, pipeline stalls for cache line
> > transfers, etc, etc, because heavy contention for the transfer buffer is
> > simply not going to be avoidable.
> >
>
> Yes, there is overhead. Let's say how can we tackle them one by one:
>
> * lock contention of transfer buffer *

I think it would be useful to think about exactly what type of
query/activity we are looking to improve the performance on. That way we
can understand the benefit of this proposal and take some baseline
measurements to analyse what is happening for those cases.

For example, a CREATE INDEX will require most if not all rows. Whereas a
SeqScan may have a selectivity of 0.001 or below in some cases. The
SeqScan case might also vary considerably in the amount of CPU required
to identify qualifying rows and that CPU might benefit from
parallelisation also.

e.g. A low-selectivity SeqScan is a good case for fairly simple
performance improvement, since the output from that executor node is
going to be much less than its input and there is less need to
parallelize the complete tree. A parallel slave node might then be
placed in charge of running a complete SeqScan node on an agreed section
of a table, rather than simply performing I/O for another query.

Best Regards, Simon Riggs