Re: asynchronous execution

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: robertmhaas(at)gmail(dot)com
Cc: amitdkhan(dot)pg(at)gmail(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: asynchronous execution
Date: 2016-10-25 09:21:50
Message-ID: 20161025.182150.230901487.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

This is the rebased version on the current master(-0004), and
added resowner stuff (0005) and unlikely(0006).

At Tue, 18 Oct 2016 10:30:51 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20161018(dot)103051(dot)30820907(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> > > - Errors in the executor can leak the WaitEventSet. Probably we need
> > > to modify ResourceOwners to be able to own WaitEventSets.
>
> WaitEventSet itself is not leaked but epoll-fd should be closed
> at failure. This seems doable with TRY-CATCHing in
> ExecAsyncEventLoop. (not yet)

Haha, that's a silly talk. The wait event can continue to live
when timeout and any error can happen on the way after the
that. I added an entry for wait event set to resource owner and
hang ones created in ExecAsyncEventWait to
TopTransactionResourceOwner. Currently WaitLatchOrSocket doesn't
do so not to change the current behavior. WaitEventSet doesn't
have usable identifier for resowner.c so currently I use the
address(pointer value) for the purpose. The patch 0005 does that.

> I measured performance and had the following result.
>
> t0 - SELECT sum(a) FROM <local single table>;
> pl - SELECT sum(a) FROM <4 local children>;
> pf0 - SELECT sum(a) FROM <4 foreign children on single connection>;
> pf1 - SELECT sum(a) FROM <4 foreign children on dedicate connections>;
>
> The result is written as "time<ms> (std dev <ms>)"
>
> sync
> t0: 3820.33 ( 1.88)
> pl: 1608.59 ( 12.06)
> pf0: 7928.29 ( 46.58)
> pf1: 8023.16 ( 26.43)
>
> async
> t0: 3806.31 ( 4.49) 0.4% faster (should be error)
> pl: 1629.17 ( 0.29) 1.3% slower
> pf0: 6447.07 ( 25.19) 18.7% faster
> pf1: 1876.80 ( 47.13) 76.6% faster
>
> t0 is not affected since the ExecProcNode stuff has gone.
>
> pl is getting a bit slower. (almost the same to simple seqscan of
> the previous patch) This should be a misprediction penalty.

Using likely macro for ExecAppend, and it seems to have shaken
off the degradation.

sync
t0: 3919.49 ( 5.95)
pl: 1637.95 ( 0.75)
pf0: 8304.20 ( 43.94)
pf1: 8222.09 ( 28.20)

async
t0: 3885.84 ( 40.20) 0.86% faster (should be error but stable on my env..)
pl: 1617.20 ( 3.51) 1.26% faster (ditto)
pf0: 6680.95 (478.72) 19.5% faster
pf1: 1886.87 ( 36.25) 77.1% faster

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
0001-robert-s-2nd-framework.patch text/x-patch 42.9 KB
0002-Fix-some-bugs.patch text/x-patch 16.3 KB
0003-Modify-async-execution-infrastructure.patch text/x-patch 29.6 KB
0004-Make-postgres_fdw-async-capable.patch text/x-patch 43.2 KB
0005-Use-resource-owner-to-prevent-wait-event-set-from-le.patch text/x-patch 8.9 KB
0006-Apply-unlikely-to-suggest-synchronous-route-of-ExecA.patch text/x-patch 1.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2016-10-25 09:23:48 Re: Radix tree for character conversion
Previous Message Amit Kapila 2016-10-25 08:15:52 Re: Patch: Implement failover on libpq connect level.