Re: CustomScan under the Gather node?

On Tue, Jan 26, 2016 at 1:30 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> What enhancement will be necessary to implement similar feature of
> partial seq-scan using custom-scan interface?
>
> It seems to me callbacks on the three points below are needed.
> * ExecParallelEstimate
> * ExecParallelInitializeDSM
> * ExecParallelInitializeWorker
>
> Anything else?
> Does ForeignScan also need equivalent enhancement?

For postgres_fdw, running the query from a parallel worker would
change the transaction semantics. Suppose you begin a transaction,
UPDATE data on the foreign server, and then run a parallel query. If
the leader performs the ForeignScan it will see the uncommitted
UPDATE, but a worker would have to make its own connection which not
be part of the same transaction and which would therefore not see the
update. That's a problem.

Also, for postgres_fdw, and many other FDWs I suspect, the assumption
is that most of the work is being done on the remote side, so doing
the work in a parallel worker doesn't seem super interesting. Instead
of incurring transfer costs to move the data from remote to local, we
incur two sets of transfer costs: first remote to local, then worker
to leader. Ouch. I think a more promising line of inquiry is to try
to provide asynchronous execution when we have something like:

Append
-> Foreign Scan
-> Foreign Scan

...so that we can return a row from whichever Foreign Scan receives
data back from the remote server first.

So it's not impossible that an FDW author could want this, but mostly
probably not. I think.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company