Re: asynchronous execution

Hello,

I cannot respond until next Monday, so I move this to the next CF
by myself.

At Tue, 15 Nov 2016 20:25:13 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20161115(dot)202513(dot)268072050(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> Hello, this is a maintenance post of reased patches.
> I added a change of ResourceOwnerData missed in 0005.
>
> At Mon, 31 Oct 2016 10:39:12 +0900 (JST), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20161031(dot)103912(dot)217430542(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> > This a PoC patch of asynchronous execution feature, based on a
> > executor infrastructure Robert proposed. These patches are
> > rebased on the current master.
> >
> > 0001-robert-s-2nd-framework.patch
> >
> > Roberts executor async infrastructure. Async-driver nodes
> > register its async-capable children and sync and data transfer
> > are done out of band of ordinary ExecProcNode channel. So async
> > execution no longer disturbs async-unaware node and slows them
> > down.
> >
> > 0002-Fix-some-bugs.patch
> >
> > Some fixes for 0001 to work. This is just to preserve the shape
> > of 0001 patch.
> >
> > 0003-Modify-async-execution-infrastructure.patch
> >
> > The original infrastructure doesn't work when multiple foreign
> > tables is on the same connection. This makes it work.
> >
> > 0004-Make-postgres_fdw-async-capable.patch
> >
> > Makes postgres_fdw to work asynchronously.
> >
> > 0005-Use-resource-owner-to-prevent-wait-event-set-from-le.patch
> >
> > This addresses a problem pointed by Robers about 0001 patch,
> > that WaitEventSet used for async execution can leak by errors.
> >
> > 0006-Apply-unlikely-to-suggest-synchronous-route-of-ExecA.patch
> >
> > ExecAppend gets a bit slower by penalties of misprediction of
> > branches. This fixes it by using unlikely() macro.
> >
> > 0007-Add-instrumentation-to-async-execution.patch
> >
> > As the description above for 0001, async infrastructure conveys
> > tuples outside ExecProcNode channel so EXPLAIN ANALYZE requires
> > special treat to show sane results. This patch tries that.
> >
> >
> > A result of a performance measurement is in this message.
> >
> > https://www.postgresql.org/message-id/20161025.182150.230901487.horiguchi.kyotaro@lab.ntt.co.jp
> >
> >
> > | t0 - SELECT sum(a) FROM <local single table>;
> > | pl - SELECT sum(a) FROM <4 local children>;
> > | pf0 - SELECT sum(a) FROM <4 foreign children on single connection>;
> > | pf1 - SELECT sum(a) FROM <4 foreign children on dedicate connections>;
> > ...
> > | async
> > | t0: 3885.84 ( 40.20) 0.86% faster (should be error but stable on my env..)
> > | pl: 1617.20 ( 3.51) 1.26% faster (ditto)
> > | pf0: 6680.95 (478.72) 19.5% faster
> > | pf1: 1886.87 ( 36.25) 77.1% faster

--
Kyotaro Horiguchi
NTT Open Source Software Center