Re: Foreign join pushdown vs EvalPlanQual

> On 2015/11/09 13:40, Kouhei Kaigai wrote:
> >> Having said that, as I said previously, I don't see much value in adding
> >> the callback routine, to be honest. I know KaiGai-san considers that
> >> that would be useful for custom joins, but I don't think that that would
> >> be useful even for foreign joins, because I think that in case of
> >> foreign joins, the practical implementation of that routine in FDWs
> >> would be to create a secondary plan and execute that plan by performing
> >> ExecProcNode, as my patch does [1]. Maybe I'm missing something, though.
>
> > I've never denied that alternative local sub-plan is one of the best
> > approach for postgres_fdw, however, I've also never heard why you can
> > say the best approach for postgres_fdw is definitely also best for
> > others.
> > If we would justify less flexible interface specification because of
> > comfort for a particular extension, it should not be an extension,
> > but a built-in feature.
> > My standpoint has been consistent through the discussion; we can never
> > predicate which feature shall be implemented on FDW interface, therefore,
> > we also cannot predicate which implementation is best for EPQ rechecks
> > also. Only FDW driver knows which is the "best" for them, not us.
>
> What the RecheckForeignScan routine does for the foreign-join case would
> be the following for tuples stored in estate->es_epqTuple[]:
>
> 1. Apply relevant restriction clauses, including fdw_recheck_quals, to
> the tuples for the baserels involved in a foreign-join, and see if the
> tuples still pass the clauses.
>
It depends on how FDW driver has restriction clauses, but you should not
use fdw_recheck_quals to recheck individual base relations, because it is
initialized to run on the joined tuple according to fdw_scan_tlist, so
restriction clauses has to be kept in other private field.

> 2. If so, form a join tuple, while applying relevant join clauses to the
> tuples, and set the join tuple in the given slot. Else set empty.
>
No need to form a joined tuple after the rechecks of base relations's
clauses. If FDW support only inner-join, it can reconstruct a joined
tuple first, then run fdw_recheck_quals (by caller) that contains both
relation's clauses and join clause.
FDW driver can choose its comfortable way according to its implementation
and capability.

> I think these would be more efficiently processed internally in core
> than externally in FDWs. That's why I don't see much value in adding
> the routine. I have to admit that that means no flexibility, though.
>
Something like "efficiently", "better", "reasonable" and etc... are
your opinions from your standpoint. Things important is why you
thought X is better and Y is worse. It is what I've wanted to see for
three months, but never seen.

Discussion will become unproductive without understanding of the
reason of different conclusion. Please don't omit why you think it
is "efficient" that can justify to enforce all FDW drivers
a particular implementation manner, as a part of interface contract.

> However, the routine as-is doesn't seem good enough, either. For
> example, since the routine is called after each of the tuples was
> re-fetched from the remote end or re-computed from the whole-row var and
> stored in the corresponding estate->es_epqTuple[], the routine wouldn't
> allow for what Robert proposed in [2]. To do such a thing, I think we
> would probably need to change the existing EPQ machinery more
> drastically and rethink the right place for calling the routine.
>
Please also see my message:
http://www.postgresql.org/message-id/9A28C8860F777E439AA12E8AEA7694F8011617C6@BPXM15GP.gisp.nec.co.jp

And, why Robert thought here is a tough challenge:
http://www.postgresql.org/message-id/CA+TgmoY5Lf+vYy1Bha=U7__S3qtMQP7d+gSSfd+LN4Xz6Fybkg@mail.gmail.com

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>