Re: Foreign join pushdown vs EvalPlanQual

From: Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>
To: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>, Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, Shigeru Hanada <shigeru(dot)hanada(at)gmail(dot)com>
Subject: Re: Foreign join pushdown vs EvalPlanQual
Date: 2015-11-26 05:04:32
Message-ID: 9A28C8860F777E439AA12E8AEA7694F801176205@BPXM15GP.gisp.nec.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> -----Original Message-----
> From: Etsuro Fujita [mailto:fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp]
> Sent: Tuesday, November 24, 2015 12:45 PM
> To: Robert Haas; Kaigai Kouhei(海外 浩平)
> Cc: Tom Lane; Kyotaro HORIGUCHI; pgsql-hackers(at)postgresql(dot)org; Shigeru Hanada
> Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
>
> On 2015/11/24 2:41, Robert Haas wrote:
> > On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> >> One subplan means FDW driver run an entire join sub-tree with local
> >> alternative sub-plan; that is my expectation for the majority case.
>
> > What I'm imagining is that we'd add handling that allows the
> > ForeignScan to have inner and outer children. If the FDW wants to
> > delegate the EvalPlanQual handling to a local plan, it can use the
> > outer child for that. Or the inner one, if it likes. The other one
> > is available for some other purposes which we can't imagine yet. If
> > this is too weird, we can only add handling for an outer subplan and
> > forget about having an inner subplan for now. I just thought to make
> > it symmetric, since outer and inner subplans are pretty deeply baked
> > into the structure of the system.
>
> I'd vote for only allowing an outer subplan.
>
The attached patch adds: Path *fdw_outerpath field to ForeignPath node.
FDW driver can set arbitrary but one path-node here.
After that, this path-node shall be transformed to plan-node by
createplan.c, then passed to FDW driver using GetForeignPlan callback.
We expect FDW driver set this plan-node on lefttree (a.k.a outerPlan).
The Plan->outerPlan is a common field, so patch size become relatively
small. FDW driver can initialize this plan at BeginForeignScan, then
execute this sub-plan-tree on demand.

Remaining portions are as previous version. ExecScanFetch is revised
to call recheckMtd always when scanrelid==0, then FDW driver can get
control using RecheckForeignScan callback.
It allows FDW driver to handle (1) EPQ recheck on underlying scan nodes,
(2) reconstruction of joined tuple, and (3) EPQ recheck on join clauses,
by its preferable implementation - including execution of an alternative
sub-plan.

> There seems to be no changes to make_foreignscan. Is that OK?
>
create_foreignscan_path(), not only make_foreignscan().

This patch is not tested by actual FDW extensions, so it is helpful
to enhance postgres_fdw to run the alternative sub-plan on EPQ recheck.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

Attachment Content-Type Size
pgsql-fdw-epq-recheck.v4.patch application/octet-stream 10.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro HORIGUCHI 2015-11-26 05:19:24 Re: [Proposal] Table partition + join pushdown
Previous Message Tom Lane 2015-11-26 04:49:07 Re: What .gitignore files do in the tarball?