Re: Foreign join pushdown vs EvalPlanQual

> > In addition, you may misunderstand the proposition of mine above.
> > You can check RelOptInfo->fdw_private on top of the GetForeignJoinPaths,
> > then, if it is second or later invocation, you can check cost of the
> > alternative path kept in the ForeignPath node previously constructed.
> > If cheapest_total_path at the moment of GetForeignJoinPaths invocation
> > is cheaper than the saved alternative path, you can adjust the node to
> > replace the alternative path node.
>
> To get the (probably unparameterized) cheapest_total_path, IIUC, we need
> to do set_cheapest during GetForeignJoinPaths in each subsequent
> invocation of that routine, don't we? And set_cheapest is expensive,
> isn't it?
>
add_path() usually drop paths that are obviously lesser than others,
so walk on join->pathlist shall have reasonable length.
Even though it has hundreds items on the pathlist, you CAN implement
EPQ fallback using alternative built-in logic.

> >>>> 2. Save the alternative Plan nodes on fdw_plans or lefttree/righttree
> >>>> somewhere you like at the GetForeignPlan()
> >>>> 3. Makes BeginForeignScan() to call ExecInitNode() towards the plan node
> >>>> saved at (2), then save the PlanState on fdw_ps, lefttree/righttree,
> >>>> or somewhere private area if not displayed on EXPLAIN.
> >>>> 4. Implement ForeignRecheck() routine. If scanrelid==0, it kicks the
> >>>> planstate node saved at (3) to generate tuple slot. Then, call the
> >>>> ExecQual() to check qualifiers being pushed down.
> >>>> 5. Makes EndForeignScab() to call ExecEndNode() towards the PlanState
> >>>> saved at (3).
>
> >> but the design that you proposed
> >> looks complicated beyond necessity. I think we should add an FDW API
> >> for doing something if FDWs have more knowledge about doing that than
> >> the core, but in your proposal, instead of the core, an FDW has to
> >> eventually do a lot of the core's work: ExecInitNode, ExecProcNode,
> >> ExecQual, ExecEndNode and so on.
>
> > It is a trade-off problem between interface flexibility and code smallness
> > of FDW extension if it fits scope of the core support.
> > I stand on the viewpoint that gives highest priority on the flexibility,
> > especially, in case when unpredictable type of modules are expected.
> > Your proposition is comfortable to FDW on behalf of RDBMS, however, nobody
> > can promise it is beneficial to FDW on behalf of columnar-store for example.
>
> Maybe I'm missing something, but why do we need such a flexiblity for
> the columnar-stores?
>
We have various kind of FDW drivers, some of use cases were unpredictable
preliminary. Our community knows 86 kind of FDW drivers in total, and only
15 of them are for RDBMS but 71 of them for other data source.
https://wiki.postgresql.org/wiki/Foreign_data_wrappers

Even if we enforce them a new interface specification comfortable to RDBMS,
we cannot guarantee it is also comfortable to other type of FDW drivers.

If module-X wants to implement the EPQ fallback routine by itself, without
alternative plan, too rich interface design prevents what module-X really
wants to do.

> > If you stick on the code smallness of FDW on behalf of RDBMS, we can add
> > utility functions on foreign.c or somewhere. It will be able to provide
> > equivalent functionality, but FDW can determine whether it use the routines.
>
> That might be an idea, but I'd like to hear the opinions of others.
>

--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>