Re: Foreign join pushdown vs EvalPlanQual

> -----Original Message-----
> From: Kaigai Kouhei(海外 浩平)
> Sent: Sunday, November 08, 2015 12:38 AM
> To: 'Robert Haas'
> Cc: Etsuro Fujita; Tom Lane; Kyotaro HORIGUCHI; pgsql-hackers(at)postgresql(dot)org;
> Shigeru Hanada
> Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
>
> > On Fri, Nov 6, 2015 at 9:42 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > > This patch needs to be rebased.
> > > One thing different from the latest version is fdw_recheck_quals of
> > > ForeignScan was added. So, ...
> > >
> > > (1) Principle is that FDW driver knows what qualifiers were pushed down
> > > and how does it kept in the private field. So, fdw_recheck_quals is
> > > redundant and to be reverted.
> > >
> > > (2) Even though the principle is as described in (1), however,
> > > wired logic in ForeignRecheck() and fdw_recheck_quals are useful
> > > default for most of FDW drivers. So, it shall be kept and valid
> > > only if RecheckForeignScan callback is not defined.
> > >
> > > Which is better approach for the v3 patch?
> > > My preference is (1), because fdw_recheck_quals is a new feature,
> > > thus, FDW driver has to be adjusted in v9.5 more or less, even if
> > > it already supports qualifier push-down.
> > > In general, interface becomes more graceful to stick its principle.
> >
> > fdw_recheck_quals seems likely to be very convenient for FDW authors,
> > and I think ripping it out would be a terrible decision.
> >
> OK, I try to co-exist fdw_recheck_quals and RecheckForeignScan callback.
>
> > I think ForeignRecheck should first call ExecQual to test
> > fdw_recheck_quals. If it returns false, return false. If it returns
> > true, then give the FDW callback a chance, if one is defined. If that
> > returns false, return false. If we haven't yet returned false,
> > return true.
> >
> I think ExecQual on fdw_recheck_quals shall be called next to the
> RecheckForeignScan callback, because econtext->ecxt_scantuple shall
> not be reconstructed unless RecheckForeignScan callback is not called
> if scanrelid==0.
>
> If RecheckForeignScan is called prior to ExecQual, FDW driver can
> take either of two options according to its preference.
>
> (1) RecheckForeignScan callback reconstruct a joined tuple based on
> the primitive EPQ slots, but nothing are rechecked by itself.
> ForeignRecheck runs ExecQual on fdw_recheck_quals that represents
> qualifiers of base relations and join condition.
>
> (2) RecheckForeignScan callback reconstruct a joined tuple based on
> the primitive EPQ slots, then rechecks qualifiers of base relations
> and join condition by itself. It put NIL on fdw_recheck_quals, so
> ExecQual in ForeignRecheck() always true.
>
> In either case, we cannot use ExecQual prior to reconstruction of
> a joined tuple because only FDW driver knows how to reconstruct it.
> So, it means ForeignScan with scanrelid==0 always has to set NIL on
> fdw_recheck_quals, if we would put ExecQual prior to the callback.
>
The attached patch is an adjusted version of the previous one.
Even though it co-exists a new callback and fdw_recheck_quals,
the callback is kicked first as follows.

----------------<cut here>----------------
@@ -85,6 +86,18 @@ ForeignRecheck(ForeignScanState *node, TupleTableSlot *slot)

ResetExprContext(econtext);

+ /*
+ * FDW driver has to recheck visibility of EPQ tuple towards
+ * the scan qualifiers once it gets pushed down.
+ * In addition, if this node represents a join sub-tree, not
+ * a scan, FDW driver is also responsible to reconstruct
+ * a joined tuple according to the primitive EPQ tuples.
+ */
+ if (fdwroutine->RecheckForeignScan)
+ {
+ if (!fdwroutine->RecheckForeignScan(node, slot))
+ return false;
+ }
return ExecQual(node->fdw_recheck_quals, econtext, false);
}
----------------<cut here>----------------

If callback is invoked first, FDW driver can reconstruct a joined tuple
with its comfortable way, then remaining checks can be done by ExecQual
and fds_recheck_quals on the caller side.
If callback would be located on the tail, FDW driver has no choice.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>