Re: Append with naive multiplexing of FDWs

Hello Kyotaro,

>"Parallel scan" at the moment means multiple workers fetch unique

>blocks from *one* table in an arbitrated manner. In this sense

>"parallel FDW scan" means multiple local workers fetch unique bundles

>of tuples from *one* foreign table, which means it is running on a

>single session. That doesn't offer an advantage.

It maybe not "parallel FDW scan", it can be "parallel shards scan"

the local workers will pick every foreign partition to scan. I have ever

draw a picture about that you can see it in the link below.

https://www.highgo.ca/2019/08/22/parallel-foreign-scan-of-postgresql/

I think the "parallel shards scan" make sence in this way.

>If parallel query processing worked in worker-per-table mode,

>especially on partitioned tables, maybe the current FDW would work

>without much of modification. But I believe asynchronous append on

>foreign tables on a single process is far resource-effective and

>moderately faster than parallel append.

As the test result, current patch can not gain more performance when 

it returns a huge number of tuples. By "parallel shards scan" method,

it can work well, because the 'parallel' can take full use of CPUs while 

'asynchronous' can't. 

Highgo Software (Canada/China/Pakistan)

URL : http://www.highgo.ca/

EMAIL: mailto:movead(dot)li(at)highgo(dot)ca