Re: Foreign join pushdown vs EvalPlanQual

> -----Original Message-----
> From: Etsuro Fujita [mailto:fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp]
> Sent: Wednesday, November 04, 2015 5:11 PM
> To: Kaigai Kouhei(海外 浩平); Robert Haas
> Cc: Tom Lane; Kyotaro HORIGUCHI; pgsql-hackers(at)postgresql(dot)org; Shigeru Hanada
> Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
>
> On 2015/11/03 22:15, Kouhei Kaigai wrote:
> > A challenge is that junk wholerow references on behalf of ROW_MARK_COPY
> > are injected by preprocess_targetlist(). It is earlier than the main path
> > consideration by query_planner(), thus, it is not predictable how remote
> > query shall be executed at this point.
> > If ROW_MARK_COPY, base tuple image is fetched using this junk attribute.
> > So, here is two options if we allow to put joined tuple on either of
> > es_epqTuple[].
> >
> > options-1) We ignore record type definition. FDW returns a joined tuple
> > towards the whole-row reference of either of the base relations in this
> > join. The junk attribute shall be filtered out eventually and only FDW
> > driver shall see, so it is harmless to do (probably).
> > This option takes no big changes, however, we need a little brave to adopt.
> >
> > options-2) We allow FDW/CSP to adjust target-list of the relevant nodes
> > after these paths get chosen by planner. It enables to remove whole-row
> > reference of base relations and add alternative whole-row reference instead
> > if FDW/CSP can support it.
> > This feature can be relevant to target-list push-down to the remote side,
> > not only EPQ rechecks, because adjustment of target-list means we allows
> > FDW/CSP to determine which expression shall be executed locally, or shall
> > not be.
> > I think, this option is more straightforward, however, needs a little bit
> > deeper consideration, because we have to design the best hook point and
> > need to ensure how path-ification will perform.
> >
> > Therefore, I think we need two steps towards the entire solution.
> > Step-1) FDW/CSP will recheck base EPQ tuples and support local
> > reconstruction on the fly. It does not need something special
> > enhancement on the planner - so we can fix up by v9.5 release.
> > Step-2) FDW/CSP will support adjustment of target-list to add whole-row
> > reference of joined tuple instead of multiple base relations, then FDW/CSP
> > will be able to put a joined tuple on either of EPQ slot if it wants - it
> > takes a new feature enhancement, so v9.6 is a suitable timeline.
> >
> > How about your opinion towards the direction?
> > I don't want to drop extra optimization opportunity, however, we are now in
> > November. I don't have enough brave to add none-obvious new feature here.
>
> I think we need to consider a general solution that can be applied not
> only to the case where the component tables in a foreign join all use
> ROW_MARK_COPY but to the case where those tables use different rowmark
> types such as ROW_MARK_COPY and ROW_MARK_EXCLUSIVE, as I pointed out
> upthread.
>
In mixture case, FDW/CSP can choose local recheck & reconstruction based
on the EPQ tuples of base relation. Nobody enforce FDW/CSP to return
a joined tuple always even if author don't want to support the feature.
Why do you think it is not a generic solution? FDW/CSP driver "can choose"
the best solution according to its implementation and capability.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>