Re: Parallel Inserts in CREATE TABLE AS

From: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
To: "Hou, Zhijie" <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com>
Cc: Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Luc Vlaming <luc(at)swarm64(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Zhihong Yu <zyu(at)yugabyte(dot)com>
Subject: Re: Parallel Inserts in CREATE TABLE AS
Date: 2020-12-22 08:45:56
Message-ID: CALj2ACWbQ95gS0z1viQC3qFVnMGAz7dcLjno9GdZv+u9RAU9eQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Dec 22, 2020 at 12:32 PM Bharath Rupireddy
<bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> On Mon, Dec 21, 2020 at 8:16 AM Hou, Zhijie <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com> wrote:
> > The cfbost seems complains about the testcase:
> >
> > Command exited with code 1
> > perl dumpregr.pl
> > === $path ===\ndiff -w -U3 C:/projects/postgresql/src/test/regress/expected/write_parallel.out C:/projects/postgresql/src/test/regress/results/write_parallel.out
> > --- C:/projects/postgresql/src/test/regress/expected/write_parallel.out 2020-12-21 01:41:17.745091500 +0000
> > +++ C:/projects/postgresql/src/test/regress/results/write_parallel.out 2020-12-21 01:47:20.375514800 +0000
> > @@ -1204,7 +1204,7 @@
> > -> Gather (actual rows=2 loops=1)
> > Workers Planned: 3
> > Workers Launched: 3
> > - -> Parallel Seq Scan on temp2 (actual rows=0 loops=4)
> > + -> Parallel Seq Scan on temp2 (actual rows=1 loops=4)
> > Filter: (col2 < 3)
> > Rows Removed by Filter: 1
> > (14 rows)
> > @@ -1233,7 +1233,7 @@
> > -> Gather (actual rows=2 loops=1)
> > Workers Planned: 3
> > Workers Launched: 3
> > - -> Parallel Seq Scan on temp2 (actual rows=0 loops=4)
> > + -> Parallel Seq Scan on temp2 (actual rows=1 loops=4)
> > Filter: (col2 < 3)
> > Rows Removed by Filter: 1
> > (14 rows)
>
> Thanks! Looks like the explain analyze test case outputs can be
> unstable because we may not get the requested number of workers
> always. The comment before explain_parallel_append function in
> partition_prune.sql explains it well.
>
> Solution is to have a function similar to explain_parallel_append, say
> explain_parallel_inserts in write_parallel.sql and use that for all
> explain analyze cases. This will make the results consistent.
> Thoughts? If okay, I will update the test cases and post new patches.

Attaching v14 patch set that has above changes. Please consider this
for further review.

With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
v14-0001-Parallel-Inserts-in-CREATE-TABLE-AS.patch application/x-patch 33.8 KB
v14-0002-Tuple-Cost-Adjustment-for-Parallel-Inserts-in-CTAS.patch application/x-patch 12.6 KB
v14-0003-Tests-For-Parallel-Inserts-in-CTAS.patch application/x-patch 27.9 KB
v14-0004-Enable-CTAS-Parallel-Inserts-For-Append.patch application/x-patch 44.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Magnus Hagander 2020-12-22 08:51:28 Re: Better client reporting for "immediate stop" shutdowns
Previous Message Michael Paquier 2020-12-22 08:37:17 Re: Fail Fast In CTAS/CMV If Relation Already Exists To Avoid Unnecessary Rewrite, Planning Costs