RE: Parallel Inserts in CREATE TABLE AS

From: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>
To: "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>, 'Bharath Rupireddy' <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, "tanghy(dot)fnst(at)fujitsu(dot)com" <tanghy(dot)fnst(at)fujitsu(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Zhihong Yu <zyu(at)yugabyte(dot)com>, Luc Vlaming <luc(at)swarm64(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>
Subject: RE: Parallel Inserts in CREATE TABLE AS
Date: 2021-05-28 00:59:27
Message-ID: OS0PR01MB57164384314D67470A04EB8A94229@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Tsunakawa, Takayuki/綱川 貴之 <tsunakawa(dot)takay(at)fujitsu(dot)com>
Sent: Friday, May 28, 2021 8:55 AM
> To: 'Bharath Rupireddy' <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>; Hou,
> Zhijie/侯 志杰 <houzj(dot)fnst(at)fujitsu(dot)com>
> Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>; Tang, Haiying/唐 海英
> <tanghy(dot)fnst(at)fujitsu(dot)com>; PostgreSQL-development
> <pgsql-hackers(at)postgresql(dot)org>; Zhihong Yu <zyu(at)yugabyte(dot)com>; Luc
> Vlaming <luc(at)swarm64(dot)com>; Dilip Kumar <dilipbalaut(at)gmail(dot)com>;
> vignesh C <vignesh21(at)gmail(dot)com>
> Subject: RE: Parallel Inserts in CREATE TABLE AS
>
> From: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
> > I'm still not sure why the execution time with 0 workers (or serial
> > execution or no parallelism involved) on my testing system is 112 sec
> > compared to 58 sec on Hou-San's system for the same use case. Maybe
> > the testing system I'm using is not of the latest configuration compared to
> others.
>
> What's the setting of wal_level on your two's systems? I thought it could be
> that you set it to > minimal, while Hou-san set it to minimal. (I forgot the
> results of 2 and 4 workers, though.)

I think I followed the configuration that Bharath-san mentioned.
It could be the hardware's difference, because I am not using SSD.
I will try to test on SSD to see if there is some difference.

I only change the the following configuration:

shared_buffers = 40GB
max_worker_processes = 32
max_parallel_maintenance_workers = 24
max_parallel_workers = 32
synchronous_commit = off
checkpoint_timeout = 1d
max_wal_size = 24GB
min_wal_size = 15GB
autovacuum = off

Best regards,
houzj

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Nancarrow 2021-05-28 01:01:31 Re: Consider parallel for lateral subqueries with limit
Previous Message tsunakawa.takay@fujitsu.com 2021-05-28 00:54:48 RE: Parallel Inserts in CREATE TABLE AS