Re: asynchronous execution

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: robertmhaas(at)gmail(dot)com
Cc: amitdkhan(dot)pg(at)gmail(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: asynchronous execution
Date: 2016-10-31 01:39:12
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


I'm not sure this is in a sutable shape for commit fest but I
decided to register this to ride on the bus for 10.0.

> Hi, this is the 7th patch to make instrumentation work.

This a PoC patch of asynchronous execution feature, based on a
executor infrastructure Robert proposed. These patches are
rebased on the current master.


Roberts executor async infrastructure. Async-driver nodes
register its async-capable children and sync and data transfer
are done out of band of ordinary ExecProcNode channel. So async
execution no longer disturbs async-unaware node and slows them


Some fixes for 0001 to work. This is just to preserve the shape
of 0001 patch.


The original infrastructure doesn't work when multiple foreign
tables is on the same connection. This makes it work.


Makes postgres_fdw to work asynchronously.


This addresses a problem pointed by Robers about 0001 patch,
that WaitEventSet used for async execution can leak by errors.


ExecAppend gets a bit slower by penalties of misprediction of
branches. This fixes it by using unlikely() macro.


As the description above for 0001, async infrastructure conveys
tuples outside ExecProcNode channel so EXPLAIN ANALYZE requires
special treat to show sane results. This patch tries that.

A result of a performance measurement is in this message.

| t0 - SELECT sum(a) FROM <local single table>;
| pl - SELECT sum(a) FROM <4 local children>;
| pf0 - SELECT sum(a) FROM <4 foreign children on single connection>;
| pf1 - SELECT sum(a) FROM <4 foreign children on dedicate connections>;
| async
| t0: 3885.84 ( 40.20) 0.86% faster (should be error but stable on my env..)
| pl: 1617.20 ( 3.51) 1.26% faster (ditto)
| pf0: 6680.95 (478.72) 19.5% faster
| pf1: 1886.87 ( 36.25) 77.1% faster


Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
0001-robert-s-2nd-framework.patch text/x-patch 42.9 KB
0002-Fix-some-bugs.patch text/x-patch 16.3 KB
0003-Modify-async-execution-infrastructure.patch text/x-patch 29.6 KB
0004-Make-postgres_fdw-async-capable.patch text/x-patch 43.2 KB
0005-Use-resource-owner-to-prevent-wait-event-set-from-le.patch text/x-patch 8.9 KB
0006-Apply-unlikely-to-suggest-synchronous-route-of-ExecA.patch text/x-patch 1.3 KB
0007-Add-instrumentation-to-async-execution.patch text/x-patch 2.9 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Karl O. Pinc 2016-10-31 03:35:40 Re: Patch to implement pg_current_logfile() function
Previous Message Kyotaro HORIGUCHI 2016-10-31 01:15:48 Re: IF (NOT) EXISTS in psql-completion