Re: Async execution of postgres_fdw.

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, mkellycs(at)gmail(dot)com, ashutosh(dot)bapat(at)enterprisedb(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Async execution of postgres_fdw.
Date: 2015-05-09 11:07:59
Message-ID: 20150509110759.GG30322@tamriel.snowman.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom,

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > I'm all for improving performance of postgres_fdw and would like to see
> > us support sending queries off to be worked asyncronously, but starting
> > execution on the remote server during ExecInitNode is against the
> > documentated FDW API spec. I discussed exactly this issue over a year
> > ago here:
>
> > http://www.postgresql.org/message-id/20131104032604.GB2706@tamriel.snowman.net
>
> > Sadly, there weren't any direct responses to that email, but I do recall
> > having a discussion on another thread (or in person?) with Tom where we
> > ended up agreeing that we can't simply remove that requirement from the
> > docs or the API.
>
> Yeah. There are at least a couple of reasons why not:

Thanks for the reminders of those.

> Also, so far as a quick review of the actual patch goes, I would really
> like to see this lose the "PFC" wrapper layer, which accounts for 95% of
> the code churn in the patch and doesn't seem to add any actual value.
> What it does add is unchecked malloc failure conditions.

Agreed, the wrapper isn't doing anything particularly useful; I had
started out thinking that would be my first comment until it became
clear where all the performance improvement was coming from.

I've gone ahead and marked this as Rejected. The concept of async
execution of postgres_fdw is certainly still open and a worthwhile goal,
but this implementation isn't the way to achieve that.

Thanks!

Stephen

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2015-05-09 12:48:03 Re: subxcnt defined as signed integer in SnapshotData and SerializeSnapshotData
Previous Message Abhijit Menon-Sen 2015-05-09 10:06:49 Re: a fast bloat measurement tool (was Re: Measuring relation free space)