Re: WIP/PoC for parallel backup

Hi,

I did some tests a while back, and here are the results. The tests were
done to simulate
a live database environment using pgbench.

machine configuration used for this test:
Instance Type: t2.xlarge
Volume Type : io1
Memory (MiB) : 16384
vCPU # : 4
Architecture : X86_64
IOP : 16000
Database Size (GB) : 102

The setup consist of 3 machines.
- one for database instances
- one for pg_basebackup client and
- one for pgbench with some parallel workers, simulating SELECT loads.

basebackup | 4 workers | 8 Workers | 16
workers
Backup Duration(Min): 69.25 | 20.44 | 19.86 | 20.15
(pgbench running with 50 parallel client simulating SELECT load)

Backup Duration(Min): 154.75 | 49.28 | 45.27 | 20.35
(pgbench running with 100 parallel client simulating SELECT load)

On Tue, Apr 21, 2020 at 9:27 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:

> On Tue, Apr 14, 2020 at 8:07 PM Asif Rehman <asifr(dot)rehman(at)gmail(dot)com>
> wrote:
>
>>
>> I forgot to make a check for no-manifest. Fixed. Attached is the updated
>> patch.
>>
>>
> Have we done any performance testing with this patch to see the benefits?
> If so, can you point me to the results? If not, then can we perform some
> tests on large backups to see the benefits of this patch/idea?
>
> --
> With Regards,
> Amit Kapila.
> EnterpriseDB: http://www.enterprisedb.com
>

--
--
Asif Rehman
Highgo Software (Canada/China/Pakistan)
URL : www.highgo.ca