Re: Parallel copy

On Wed, Sep 16, 2020 at 1:20 PM Greg Nancarrow <gregn4422(at)gmail(dot)com> wrote:
>
> Fortunately I have been given permission to share the exact table
> definition and data I used, so you can check the behaviour and timings
> on your own test machine.
>

Thanks Greg for the script. I ran your test case and I didn't observe
any increase in exec time with 1 worker, indeed, we have benefitted a
few seconds from 0 to 1 worker as expected.

Execution time is in seconds. Each test case is executed 3 times on
release build. Each time the data directory is recreated.

Case 1: 1000000 rows, 2GB
With Patch, default configuration, 0 worker: 88.933, 92.261, 88.423
With Patch, default configuration, 1 worker: 73.825, 74.583, 72.678

With Patch, custom configuration, 0 worker: 76.191, 78.160, 78.822
With Patch, custom configuration, 1 worker: 61.289, 61.288, 60.573

Case 2: 2550000 rows, 5GB
With Patch, default configuration, 0 worker: 246.031, 188.323, 216.683
With Patch, default configuration, 1 worker: 156.299, 153.293, 170.307

With Patch, custom configuration, 0 worker: 197.234, 195.866, 196.049
With Patch, custom configuration, 1 worker: 157.173, 158.287, 157.090

[1] - Custom configuration is set up to ensure that no other processes
influence the results. The postgresql.conf used:
shared_buffers = 40GB
synchronous_commit = off
checkpoint_timeout = 1d
max_wal_size = 24GB
min_wal_size = 15GB
autovacuum = off
max_worker_processes = 32
max_parallel_maintenance_workers = 24
max_parallel_workers = 32

With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com