Re: Async execution of postgres_fdw.

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: sfrost(at)snowman(dot)net
Cc: tgl(at)sss(dot)pgh(dot)pa(dot)us, mkellycs(at)gmail(dot)com, ashutosh(dot)bapat(at)enterprisedb(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Async execution of postgres_fdw.
Date: 2015-05-11 02:37:19
Message-ID: 20150511.113719.84171254.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Hello.

> I've gone ahead and marked this as Rejected. The concept of async
> execution of postgres_fdw is certainly still open and a worthwhile goal,
> but this implementation isn't the way to achieve that.

It sounds fair. I'm satisfied that we have agreed that the goal
is worthwhile. I'll show other implementations sooner.

Thank you.

> * Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> > Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > > I'm all for improving performance of postgres_fdw and would like to see
> > > us support sending queries off to be worked asyncronously, but starting
> > > execution on the remote server during ExecInitNode is against the
> > > documentated FDW API spec. I discussed exactly this issue over a year
> > > ago here:
> >
> > > http://www.postgresql.org/message-id/20131104032604.GB2706@tamriel.snowman.net
> >
> > > Sadly, there weren't any direct responses to that email, but I do recall
> > > having a discussion on another thread (or in person?) with Tom where we
> > > ended up agreeing that we can't simply remove that requirement from the
> > > docs or the API.
> >
> > Yeah. There are at least a couple of reasons why not:
>
> Thanks for the reminders of those.
>
> > Also, so far as a quick review of the actual patch goes, I would really
> > like to see this lose the "PFC" wrapper layer, which accounts for 95% of
> > the code churn in the patch and doesn't seem to add any actual value.
> > What it does add is unchecked malloc failure conditions.
>
> Agreed, the wrapper isn't doing anything particularly useful; I had
> started out thinking that would be my first comment until it became
> clear where all the performance improvement was coming from.
>
> I've gone ahead and marked this as Rejected. The concept of async
> execution of postgres_fdw is certainly still open and a worthwhile goal,
> but this implementation isn't the way to achieve that.

--
Kyotaro Horiguchi
NTT Open Source Software Center

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2015-05-11 02:51:33 Re: Custom/Foreign-Join-APIs (Re: [v9.5] Custom Plan API)
Previous Message Andres Freund 2015-05-11 02:25:48 Re: Custom/Foreign-Join-APIs (Re: [v9.5] Custom Plan API)