Re: Custom/Foreign-Join-APIs (Re: [v9.5] Custom Plan API)

Let me remind the problem not to be forgotten towards v9.5.

The commit 1a8a4e5cde2b7755e11bde2ea7897bd650622d3e disallowed extensions
to call create_plan_recurse(), however, it is required for custom-scan node
that implements own join logic and takes child nodes to construct Plan node
from Path node. So, at this moment, we cannot utilize the new feature well
unless extension copies & pastes createplan.c to its own source tree (ugly!).

Tom suggested an alternative infrastructure that tells planner Path nodes
to be passed to create_plan_recurse() in createplan.c.

> > A possible compromise that we could perhaps still wedge into 9.5 is to
> > extend CustomPath with a List of child Paths, and CustomScan with a List
> > of child Plans, which createplan.c would know to build from the Paths,
> > and other modules would then also be aware of these children. I find that
> > uglier than a separate join node type, but it would be tolerable I guess.
> >
> The attached patch implements what you suggested as is.
> It allows custom-scan providers to have child Paths without exporting
> create_plan_recurse(), and enables to represent N-way join naturally.
> Please add any solution, even if we don't reach the consensus of how
> create_plan_recurse (and other useful static functions) are visible to
> extensions.
>
Then, I made a patch (which was attached on the last message) according to
the suggestion. I think it is one possible option.

Or, one other option is to revert create_plan_recurse() non-static function
as the infrastructure originally designed.

I expect people think it is not a graceful design to force extensions to
copy and paste core file with small adjustment. So, either of options, or
others if any, needs to be merged to solve the problem.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org
> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Kouhei Kaigai
> Sent: Friday, May 15, 2015 8:44 AM
> To: Tom Lane; Kohei KaiGai
> Cc: Robert Haas; Thom Brown; Shigeru Hanada; pgsql-hackers(at)postgreSQL(dot)org
> Subject: Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)
>
> > A possible compromise that we could perhaps still wedge into 9.5 is to
> > extend CustomPath with a List of child Paths, and CustomScan with a List
> > of child Plans, which createplan.c would know to build from the Paths,
> > and other modules would then also be aware of these children. I find that
> > uglier than a separate join node type, but it would be tolerable I guess.
> >
> The attached patch implements what you suggested as is.
> It allows custom-scan providers to have child Paths without exporting
> create_plan_recurse(), and enables to represent N-way join naturally.
> Please add any solution, even if we don't reach the consensus of how
> create_plan_recurse (and other useful static functions) are visible to
> extensions.
>
>
> Patch detail:
> It adds a List field (List *custom_children) to CustomPath structure
> to inform planner its child Path nodes, to be transformed to Plan node
> through the planner's job.
> CustomScan also have a new List field to have its child Plan nodes
> which shall be processed by setrefs.c and subselect.c.
> PlanCustomPath callback was extended to have a list of Plan nodes
> that were constructed on create_customscan_plan in core, it is
> a job of custom-scan provider to attach these Plan nodes onto
> lefttree, righttree or the custom_children list.
> CustomScanState also have an array to have PlanState nodes of the
> children. It is used for EXPLAIN command know the child nodes.
>
> Regarding of FDW, as Hanada-san mentioned, I'm uncertain whether
> similar feature is also needed because its join-pushdown feature
> scan on the result-set of remotely joined relations, thus no need
> to have local child Path nodes.
> So, I put this custom_children list on CustomXXXX structure only.
>
> It may need additional section in the documentation.
>
> Thanks,
> --
> NEC Business Creation Division / PG-Strom Project
> KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>