Re: asynchronous execution

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: robertmhaas(at)gmail(dot)com
Cc: amitdkhan(dot)pg(at)gmail(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: asynchronous execution
Date: 2016-10-31 01:39:12
Message-ID: 20161031.103912.217430542.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Hello,

I'm not sure this is in a sutable shape for commit fest but I
decided to register this to ride on the bus for 10.0.

> Hi, this is the 7th patch to make instrumentation work.

This a PoC patch of asynchronous execution feature, based on a
executor infrastructure Robert proposed. These patches are
rebased on the current master.

0001-robert-s-2nd-framework.patch

Roberts executor async infrastructure. Async-driver nodes
register its async-capable children and sync and data transfer
are done out of band of ordinary ExecProcNode channel. So async
execution no longer disturbs async-unaware node and slows them
down.

0002-Fix-some-bugs.patch

Some fixes for 0001 to work. This is just to preserve the shape
of 0001 patch.

0003-Modify-async-execution-infrastructure.patch

The original infrastructure doesn't work when multiple foreign
tables is on the same connection. This makes it work.

0004-Make-postgres_fdw-async-capable.patch

Makes postgres_fdw to work asynchronously.

0005-Use-resource-owner-to-prevent-wait-event-set-from-le.patch

This addresses a problem pointed by Robers about 0001 patch,
that WaitEventSet used for async execution can leak by errors.

0006-Apply-unlikely-to-suggest-synchronous-route-of-ExecA.patch

ExecAppend gets a bit slower by penalties of misprediction of
branches. This fixes it by using unlikely() macro.

0007-Add-instrumentation-to-async-execution.patch

As the description above for 0001, async infrastructure conveys
tuples outside ExecProcNode channel so EXPLAIN ANALYZE requires
special treat to show sane results. This patch tries that.

A result of a performance measurement is in this message.

https://www.postgresql.org/message-id/20161025.182150.230901487.horiguchi.kyotaro@lab.ntt.co.jp

| t0 - SELECT sum(a) FROM <local single table>;
| pl - SELECT sum(a) FROM <4 local children>;
| pf0 - SELECT sum(a) FROM <4 foreign children on single connection>;
| pf1 - SELECT sum(a) FROM <4 foreign children on dedicate connections>;
...
| async
| t0: 3885.84 ( 40.20) 0.86% faster (should be error but stable on my env..)
| pl: 1617.20 ( 3.51) 1.26% faster (ditto)
| pf0: 6680.95 (478.72) 19.5% faster
| pf1: 1886.87 ( 36.25) 77.1% faster

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
0001-robert-s-2nd-framework.patch text/x-patch 42.9 KB
0002-Fix-some-bugs.patch text/x-patch 16.3 KB
0003-Modify-async-execution-infrastructure.patch text/x-patch 29.6 KB
0004-Make-postgres_fdw-async-capable.patch text/x-patch 43.2 KB
0005-Use-resource-owner-to-prevent-wait-event-set-from-le.patch text/x-patch 8.9 KB
0006-Apply-unlikely-to-suggest-synchronous-route-of-ExecA.patch text/x-patch 1.3 KB
0007-Add-instrumentation-to-async-execution.patch text/x-patch 2.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Karl O. Pinc 2016-10-31 03:35:40 Re: Patch to implement pg_current_logfile() function
Previous Message Kyotaro HORIGUCHI 2016-10-31 01:15:48 Re: IF (NOT) EXISTS in psql-completion