Re: Parallel Seq Scan

On Mon, 2015-07-06 at 10:37 +0530, Amit Kapila wrote:

> Or the other way to look at it could be separate out fields which are
> required for parallel scan which is done currently by forming a
> separate structure ParallelHeapScanDescData.
>
I was suggesting that you separate out both the normal scan fields and
the partial scan fields, that way we're sure that rs_nblocks is not
accessed during a parallel scan.

Or, you could try wrapping the parts of heapam.c that are affected by
parallelism into new static functions.

> The reason why partial scan can't be mixed with sync scan is that in
> parallel
> scan, it performs the scan of heap by synchronizing blocks (each
> parallel worker
> scans a block and then asks for a next block to scan) among parallel
> workers.
> Now if we try to make sync scans work along with it, the
> synchronization among
> parallel workers will go for a toss. It might not be impossible to
> make that
> work in some way, but not sure if it is important enough for sync
> scans to work
> along with parallel scan.

I haven't tested it, but I think it would still be helpful. The block
accesses are still in order even during a partial scan, so why wouldn't
it help?

You might be concerned about the reporting of a block location, which
would become more noisy with increased parallelism. But in my original
testing, sync scans weren't very sensitive to slight deviations, because
of caching effects.

> tqueue.c is mainly designed to pass tuples between parallel workers
> and currently it is used in Funnel operator to gather the tuples
> generated
> by all the parallel workers. I think we can use it for any other
> operator
> which needs tuple communication among parallel workers.

Some specifics of the Funnel operator seem to be a part of tqueue, which
doesn't make sense to me. For instance, reading from the set of queues
in a round-robin fashion is part of the Funnel algorithm, and doesn't
seem suitable for a generic tuple communication mechanism (that would
never allow order-sensitive reading, for example).

Regards,
Jeff Davis