Re: Foreign join pushdown vs EvalPlanQual

Hello,

At Thu, 5 Nov 2015 01:58:00 +0000, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote in <9A28C8860F777E439AA12E8AEA7694F80116284C(at)BPXM15GP(dot)gisp(dot)nec(dot)co(dot)jp>
> > So, as the third way, I propose to resurrect the abandoned
> > ForeinJoinState seems to be for the unearthed requirements. FDW
> > returns ForeignJoinPath, not ForeignScanPath then finally it
> > becomes ForeignJoinState, which is handeled as a join node with
> > no doubt.
> >
> > What do you think about this?
> >
> Apart from EPQ issues, it is fundamentally impossible to reflect
> the remote join tree on local side, because remote server runs
> the partial join in their best or arbitrary way.
> If this ForeignJoinState has just a compatible join sub-tree, what
> is the difference from the alternative local join sub-plan?

I think the ForeignJoinState don't have subnodes and might has no
difference in its structure from ForeignScanState. Its
significant difference from ForeignScanState would be that the
core can properly handle the return from the node as a joined
tuple in ordinary way. Executor no more calls ExecScan for joined
tuples again.

> Even if we have another node, the roles of FDW driver is unchanged.
> It eventually needs to do them:
> 1. Recheck scan-qualifier of base foreign table
> 2. Recheck join-clause of remote joins
> 3. Reconstruct a joined tuple

Yes, the most significant point of this proposal is in not FDW
side but core side.

> I try to estimate your intention...
> You say that ForeignScan with scanrelid==0 is not a scan actually,
> so it is problematic to call ExecScan on ExecForeignScan always.
> Thus, individual ForeignJoin shall be defined.
> Right?

Definitely.

> In case of scanrelid==0, it performs like a scan on pseudo relation
> that has record type defined by fdw_scan_tlist. The rows generated
> with this node are consists of rows in underlying base relations.
> A significant point is, FDW driver is responsible to generate the
> rows according to the fdw_scan_tlist. Once FDW driver generates rows,
> ExecScan() runs remaining tasks - execution of host clauses (although
> it is not easy to image remote join includes host clause has cheaper
> cost than others) and projection.

Agreed. The role of FDW won't be changed by introducing
ForeignJoin.

> One thing I can agree is, ForeignScan is enforced to use ExecScan,
> thus some FDW driver may concern about this hard-wired logic.
> If we try to make ForeignScan unbound from the ExecScan, I like to
> suggest to revise ExecForeignScan, just invoke a callback; then
> FDW driver can choose whether ExecScan is best or not.

Agreed. Calling ExecScan unconditionally from ForeignScan is the
cause of the root(?) cause I mentioned. Since there'd be no
difference in data structure between Foreign(Join&Node), calling
fdwroutine->ExecForeignScan() or something instaed of ExecScan()
from ExecForeignScan could be the alternative and most promising
solution for all problems in focus now.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center