Re: Custom/Foreign-Join-APIs (Re: [v9.5] Custom Plan API)

> Where are we on this? AFAIK, we have now a feature with no documentation
> and no example in-core to test those custom routine APIs, hence moved to
> next CF.
>
Now Hanada-san is working on the example module that use this new
infrastructure on top of postgres_fdw. Probably, he will submit the
patch within a couple of days, for the upcoming commit fest.

Regarding to the documentation, a consensus was to make up a wikipage
to edit the description by everyone, then it shall become source of
SGML file.
The latest one is here:
https://wiki.postgresql.org/wiki/CustomScanInterface

Anyway, the next commit-fest shall within a couple of days. I'd like to
have discussion for the feature.

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

> -----Original Message-----
> From: Michael Paquier [mailto:michael(dot)paquier(at)gmail(dot)com]
> Sent: Friday, February 13, 2015 4:38 PM
> To: Kaigai Kouhei(海外 浩平)
> Cc: Robert Haas; Tom Lane; pgsql-hackers(at)postgreSQL(dot)org; Shigeru Hanada
> Subject: ##freemail## Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5]
> Custom Plan API)
>
>
>
> On Thu, Jan 15, 2015 at 8:02 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
>
>
> > On Fri, Jan 9, 2015 at 10:51 AM, Kouhei Kaigai
> <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > > When custom-scan node replaced a join-plan, it shall have at
> least two
> > > child plan-nodes. The callback handler of PlanCustomPath needs
> to be
> > > able to call create_plan_recurse() to transform the underlying
> > > path-nodes to plan-nodes, because this custom-scan node may
> take other
> > > built-in scan or sub-join nodes as its inner/outer input.
> > > In case of FDW, it shall kick any underlying scan relations
> to remote
> > > side, thus we may not expect ForeignScan has underlying plans...
> >
> > Do you have an example of this?
> >
> Yes, even though full code set is too large for patch submission...
>
> https://github.com/pg-strom/devel/blob/master/src/gpuhashjoin.
> c#L1880
>
> This create_gpuhashjoin_plan() is PlanCustomPath callback of
> GpuHashJoin.
> It takes GpuHashJoinPath inherited from CustomPath that has
> multiple
> underlying scan/join paths.
> Once it is called back from the backend, it also calls
> create_plan_recurse()
> to make inner/outer plan nodes according to the paths.
>
> In the result, we can see the following query execution plan that
> CustomScan
> takes underlying scan plans.
>
> postgres=# EXPLAIN SELECT * FROM t0 NATURAL JOIN t1 NATURAL JOIN
> t2;
> QUERY PLAN
> --------------------------------------------------------------
> --------------------
> Custom Scan (GpuHashJoin) (cost=2968.00..140120.31
> rows=3970922 width=143)
> Hash clause 1: (aid = aid)
> Hash clause 2: (bid = bid)
> Bulkload: On
> -> Custom Scan (GpuScan) on t0 (cost=500.00..57643.00
> rows=4000009 width=77)
> -> Custom Scan (MultiHash) (cost=734.00..734.00 rows=40000
> width=37)
> hash keys: aid
> nBatches: 1 Buckets: 46000 Memory Usage: 99.99%
> -> Seq Scan on t1 (cost=0.00..734.00 rows=40000
> width=37)
> -> Custom Scan (MultiHash) (cost=734.00..734.00
> rows=40000 width=37)
> hash keys: bid
> nBatches: 1 Buckets: 46000 Memory Usage: 49.99%
> -> Seq Scan on t2 (cost=0.00..734.00 rows=40000
> width=37)
> (13 rows)
>
>
>
> Where are we on this? AFAIK, we have now a feature with no documentation
> and no example in-core to test those custom routine APIs, hence moved to
> next CF.
> --
>
> Michael