Re: Foreign join pushdown vs EvalPlanQual

> At Thu, 26 Nov 2015 05:04:32 +0000, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote
> in <9A28C8860F777E439AA12E8AEA7694F801176205(at)BPXM15GP(dot)gisp(dot)nec(dot)co(dot)jp>
> > > On 2015/11/24 2:41, Robert Haas wrote:
> > > > On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > > >> One subplan means FDW driver run an entire join sub-tree with local
> > > >> alternative sub-plan; that is my expectation for the majority case.
> > >
> > > > What I'm imagining is that we'd add handling that allows the
> > > > ForeignScan to have inner and outer children. If the FDW wants to
> > > > delegate the EvalPlanQual handling to a local plan, it can use the
> > > > outer child for that. Or the inner one, if it likes. The other one
> > > > is available for some other purposes which we can't imagine yet. If
> > > > this is too weird, we can only add handling for an outer subplan and
> > > > forget about having an inner subplan for now. I just thought to make
> > > > it symmetric, since outer and inner subplans are pretty deeply baked
> > > > into the structure of the system.
> > >
> > > I'd vote for only allowing an outer subplan.
> > >
> > The attached patch adds: Path *fdw_outerpath field to ForeignPath node.
> > FDW driver can set arbitrary but one path-node here.
>
> It is named "outerpath/plan". Surely we used the term 'outer' in
> association with other nodes for disign decision but is it valid
> to call it outer? Addition to that, there's no innerpath in this
> patch and have "path" instead.
>
Just "path" is too simple, not to inform people the expected usage
of the node.
If we would assign another name, my preference is "fdw_subpath" or
"fdw_altpath".

> > After that, this path-node shall be transformed to plan-node by
> > createplan.c, then passed to FDW driver using GetForeignPlan callback.
> > We expect FDW driver set this plan-node on lefttree (a.k.a outerPlan).
> > The Plan->outerPlan is a common field, so patch size become relatively
>
> Plan->outerPlan => Plan->lefttree?
>
Yes, s/outerPlan/lefttree/g

> > small. FDW driver can initialize this plan at BeginForeignScan, then
> > execute this sub-plan-tree on demand.
> >
> > Remaining portions are as previous version. ExecScanFetch is revised
> > to call recheckMtd always when scanrelid==0, then FDW driver can get
> > control using RecheckForeignScan callback.
>
> Perhaps we need a comment about foreignscan as a fake join for
> the case with scanrelid == 0 in ExecScanReScan.
>
Indeed,

> > It allows FDW driver to handle (1) EPQ recheck on underlying scan nodes,
> > (2) reconstruction of joined tuple, and (3) EPQ recheck on join clauses,
> > by its preferable implementation - including execution of an alternative
> > sub-plan.
>
> In ForeignRecheck, ExecQual on fdw_recheck_quals is executed if
> RecheckForeignScan returns true, which I think the signal that
> the returned tuple matches the recheck qual. Whether do you think
> have the responsibility to check the reconstructed tuple when
> RecheckCoreignScan is registered, RecheckForeignScan or ExecQual?
>
Only RecheckForeignScan can reconstruct a joined tuple. On the other
hands, both of facility can recheck scan-qualifiers and join-clauses.
FDW author can choose its design according to his preference.
If fdw_recheck_quals==NIL, FDW can apply all the rechecks within
RecheckForeignScan callback.

> The documentation says as following so I think the former has.
>
> # I don't understhad what 'can or must' means, though... 'can and
> # must'?
>
> + Also, this callback can or must recheck scan qualifiers and join
> + conditions which are pushed down. Especially, it needs special
>
If fdw_recheck_quals is set up correctly and join type is inner join,
FDW driver does not recheck by itself. Elsewhere, it has to recheck
the joined tuple, not only reconstruction.
I try to revise the SGML stuff.

>
> > > There seems to be no changes to make_foreignscan. Is that OK?
> > >
> > create_foreignscan_path(), not only make_foreignscan().
> >
> > This patch is not tested by actual FDW extensions, so it is helpful
> > to enhance postgres_fdw to run the alternative sub-plan on EPQ recheck.
>
> regardes,
>
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>