Re: Async execution of postgres_fdw.

Kyotaro,

* Kyotaro HORIGUCHI (horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp) wrote:
> The attached is the fixed patch. It apparently improves the
> performance for the test case shown in the previous mail, in
> which the average tuple length is about 140 bytes.

I'm all for improving performance of postgres_fdw and would like to see
us support sending queries off to be worked asyncronously, but starting
execution on the remote server during ExecInitNode is against the
documentated FDW API spec. I discussed exactly this issue over a year
ago here:

http://www.postgresql.org/message-id/20131104032604.GB2706@tamriel.snowman.net

Sadly, there weren't any direct responses to that email, but I do recall
having a discussion on another thread (or in person?) with Tom where we
ended up agreeing that we can't simply remove that requirement from the
docs or the API.

I certainly appreciate that you've put quite a bit of effort into this
but I'm afraid we can't accept it while it's starting to run a query on
the remote side during the ExecInitNode phase. The query can not start
executing on the remote side until InterateForeignScan is called.

You might consider looking at the other suggestion in that email with
regard to adding an Async mechanism to the executor. I didn't get to
the point of writing code, but I did think about it a fair bit and still
believe that could work.

I'm not going to change the status of this patch in the CommitFest at
this time, in case anyone else feels I've misunderstood or not correctly
analyzed what the patch does (I'll admit, I've only read it and not
actually compiled it or run it with a debugger, but I'm pretty sure my
reading of what's happening is correct..), but I'm afraid this is going
to have to end up as Rejected.

Thanks!

Stephen