Re: asynchronous execution

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: corey(dot)huinker(at)gmail(dot)com
Cc: tgl(at)sss(dot)pgh(dot)pa(dot)us, Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp, robertmhaas(at)gmail(dot)com, amitdkhan(dot)pg(at)gmail(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: asynchronous execution
Date: 2017-03-17 08:35:05
Message-ID: 20170317.173505.152063931.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

At Thu, 16 Mar 2017 17:16:32 -0400, Corey Huinker <corey(dot)huinker(at)gmail(dot)com> wrote in <CADkLM=cBZEX9L9HnhJYrtfiAN5Ebdu=xbvM_poWVGBR7yN3gVw(at)mail(dot)gmail(dot)com>
> On Thu, Mar 16, 2017 at 4:17 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> > Corey Huinker <corey(dot)huinker(at)gmail(dot)com> writes:
> > > I reworked the test such that all of the foreign tables inherit from the
> > > same parent table, and if you query that you do get async execution. But
> > It
> > > doesn't work when just stringing together those foreign tables with UNION
> > > ALLs.
> >
> > > I don't know how to proceed with this review if that was a goal of the
> > > patch.
> >
> > Whether it was a goal or not, I'd say there is something either broken
> > or incorrectly implemented if you don't see that. The planner (and
> > therefore also the executor) generally treats inheritance the same as
> > simple UNION ALL. If that's not the case here, I'd want to know why.
> >
> > regards, tom lane
> >
>
> Updated commitfest entry to "Returned With Feedback".

Sorry for the absense. For information, I'll continue to write
some more.

At Tue, 14 Mar 2017 10:28:36 +0900, Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp> wrote in <e7dc8128-f32b-ff9a-870e-f1117b8e4fa6(at)lab(dot)ntt(dot)co(dot)jp>
> async
> t0: 3806.31 ( 4.49) 0.4% faster (should be error)
> pl: 1629.17 ( 0.29) 1.3% slower
> pf0: 6447.07 ( 25.19) 18.7% faster
> pf1: 1876.80 ( 47.13) 76.6% faster
> </quote>
>
> IIUC, pf0 and pf1 is the same test case (all 4 foreign tables target the
> same server) measured with different implementations of the patch.

pf0 is measured on a partitioned(sharded) tables on one foreign
server, that is, sharing a connection. pf1 is in contrast sharded
tables that have dedicate server (or connection). The parent
server is async-patched and the child server is not patched.

Async-capable plan is generated in planner. An Append contains at
least one async-capable child becomes async-aware Append. So the
async feature should be effective also for the UNION ALL case.

The following will work faster than unpatched version.I

SELECT sum(a) FROM (SELECT a FROM ft10 UNION ALL SELECT a FROM ft20 UNION ALL SELECT a FROM ft30 UNION ALL SELECT a FROM ft40) as ft;

I'll measure the performance for the case next week.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
0002-Asynchronous-execution-framework.patch text/x-patch 46.7 KB
0004-Apply-unlikely-to-suggest-synchronous-route-of-ExecA.patch text/x-patch 1.3 KB
0003-Make-postgres_fdw-async-capable.patch text/x-patch 50.9 KB
0001-Allow-wait-event-set-to-be-registered-to-resource-ow.patch text/x-patch 9.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2017-03-17 08:57:23 Re: Partitioned tables and relfilenode
Previous Message Tsunakawa, Takayuki 2017-03-17 08:18:37 Re: PATCH: Make pg_stop_backup() archive wait optional