Re: Custom/Foreign-Join-APIs (Re: [v9.5] Custom Plan API)

From: Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "pgsql-hackers(at)postgreSQL(dot)org" <pgsql-hackers(at)postgresql(dot)org>, Shigeru Hanada <shigeru(dot)hanada(at)gmail(dot)com>
Subject: Re: Custom/Foreign-Join-APIs (Re: [v9.5] Custom Plan API)
Date: 2015-01-14 23:02:56
Message-ID: 9A28C8860F777E439AA12E8AEA7694F8010A1C61@BPXM15GP.gisp.nec.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On Fri, Jan 9, 2015 at 10:51 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > When custom-scan node replaced a join-plan, it shall have at least two
> > child plan-nodes. The callback handler of PlanCustomPath needs to be
> > able to call create_plan_recurse() to transform the underlying
> > path-nodes to plan-nodes, because this custom-scan node may take other
> > built-in scan or sub-join nodes as its inner/outer input.
> > In case of FDW, it shall kick any underlying scan relations to remote
> > side, thus we may not expect ForeignScan has underlying plans...
>
> Do you have an example of this?
>
Yes, even though full code set is too large for patch submission...

https://github.com/pg-strom/devel/blob/master/src/gpuhashjoin.c#L1880

This create_gpuhashjoin_plan() is PlanCustomPath callback of GpuHashJoin.
It takes GpuHashJoinPath inherited from CustomPath that has multiple
underlying scan/join paths.
Once it is called back from the backend, it also calls create_plan_recurse()
to make inner/outer plan nodes according to the paths.

In the result, we can see the following query execution plan that CustomScan
takes underlying scan plans.

postgres=# EXPLAIN SELECT * FROM t0 NATURAL JOIN t1 NATURAL JOIN t2;
QUERY PLAN
----------------------------------------------------------------------------------
Custom Scan (GpuHashJoin) (cost=2968.00..140120.31 rows=3970922 width=143)
Hash clause 1: (aid = aid)
Hash clause 2: (bid = bid)
Bulkload: On
-> Custom Scan (GpuScan) on t0 (cost=500.00..57643.00 rows=4000009 width=77)
-> Custom Scan (MultiHash) (cost=734.00..734.00 rows=40000 width=37)
hash keys: aid
nBatches: 1 Buckets: 46000 Memory Usage: 99.99%
-> Seq Scan on t1 (cost=0.00..734.00 rows=40000 width=37)
-> Custom Scan (MultiHash) (cost=734.00..734.00 rows=40000 width=37)
hash keys: bid
nBatches: 1 Buckets: 46000 Memory Usage: 49.99%
-> Seq Scan on t2 (cost=0.00..734.00 rows=40000 width=37)
(13 rows)

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

> -----Original Message-----
> From: Robert Haas [mailto:robertmhaas(at)gmail(dot)com]
> Sent: Thursday, January 15, 2015 2:07 AM
> To: Kaigai Kouhei(海外 浩平)
> Cc: Tom Lane; pgsql-hackers(at)postgreSQL(dot)org; Shigeru Hanada
> Subject: ##freemail## Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5]
> Custom Plan API)
>
> On Fri, Jan 9, 2015 at 10:51 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > When custom-scan node replaced a join-plan, it shall have at least two
> > child plan-nodes. The callback handler of PlanCustomPath needs to be
> > able to call create_plan_recurse() to transform the underlying
> > path-nodes to plan-nodes, because this custom-scan node may take other
> > built-in scan or sub-join nodes as its inner/outer input.
> > In case of FDW, it shall kick any underlying scan relations to remote
> > side, thus we may not expect ForeignScan has underlying plans...
>
> Do you have an example of this?
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL
> Company

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Merlin Moncure 2015-01-14 23:38:45 Re: hung backends stuck in spinlock heavy endless loop
Previous Message Andres Freund 2015-01-14 22:59:19 Re: s_lock.h default definitions are rather confused