Re: Parallel INSERT (INTO ... SELECT ...)

From: Greg Nancarrow <gregn4422(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Antonin Houska <ah(at)cybertec(dot)at>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Parallel INSERT (INTO ... SELECT ...)
Date: 2021-01-21 01:05:44
Message-ID: CAJcOf-cFT8ogPufhVEaK9_ty5TMA2ch5cG51QG7FpkeWOcwz=w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thanks for the feedback.
Posting an updated set of patches. Changes are based on feedback, as
detailed below:

There's a couple of potential issues currently being looked at:
- locking issues in additional parallel-safety checks?
- apparent uneven work distribution across the parallel workers, for
large insert data

[Antonin]
- Fixed bad Assert in PrepareParallelMode()
- Added missing comment to explain use of GetCurrentCommandId() in
PrepareParallelMode()
- Some variable name shortening in a few places
- Created common function for creation of non-parallel and parallel
ModifyTable paths
- Path count variable changed to bool
- Added FIXME comment to dubious code for creating Gather target-list
from ModifyTable subplan
- Fixed check on returningLists to use NIL instead of NULL

[Amit]
- Moved additional parallel-safety checks (for modify case) into
max_parallel_hazard()
- Removed redundant calls to max_parallel_hazard_test()
- Added Asserts to "should never happen" null-attribute cases (and
added WARNING log missing from one case)
- Added comment for use of NoLock in max_parallel_hazard_for_modify()

[Vignesh]
- Fixed a couple of typos
- Added a couple of test cases for testing that the same transaction
is used by all parallel workers

Regards,
Greg Nancarrow
Fujitsu Australia

Attachment Content-Type Size
v12-0001-Enable-parallel-SELECT-for-INSERT-INTO-.-SELECT.patch application/octet-stream 20.8 KB
v12-0002-Parallel-SELECT-for-INSERT-INTO-.-SELECT-tests-and-doc.patch application/octet-stream 57.3 KB
v12-0003-Enable-parallel-INSERT-and-or-SELECT-for-INSERT-INTO.patch application/octet-stream 43.8 KB
v12-0004-Parallel-INSERT-and-or-SELECT-for-INSERT-INTO-tests-and-doc.patch application/octet-stream 19.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2021-01-21 01:11:32 Re: POC: postgres_fdw insert batching
Previous Message Zhihong Yu 2021-01-21 01:02:56 Re: POC: postgres_fdw insert batching