Re: design for parallel backup

On Thu, Apr 30, 2020 at 6:06 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Thu, Apr 30, 2020 at 3:52 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > Why 8kb? That's smaller than what we currently do in pg_basebackup,
> > afaictl, and you're actually going to be bottlenecked by syscall
> > overhead at that point (unless you disable / don't have the whole intel
> > security mitigation stuff).
>
> I just picked something. Could easily try other things.

I tried changing the write size to 64kB, keeping the rest the same.
Here are the results:

filesystem media 1(at)64GB 2(at)32GB 4(at)16GB 8(at)8GB 16(at)4GB
xfs mag 65 53 64 74 79
ext4 mag 96 68 75 303 437
xfs ssd 75 43 29 33 38
ext4 ssd 96 68 63 214 254
spread spread n/a n/a 43 38 40

And here again are the previous results with an 8kB write size:

xfs mag 97 53 60 67 71
ext4 mag 94 68 66 335 549
xfs ssd 97 55 33 27 25
ext4 ssd 116 70 66 227 450
spread spread n/a n/a 48 42 44

Generally, those numbers look better than the previous numbers, but
parallelism still looks fairly appealing on the SSD storage - less so
on magnetic disks, at least in this test.

Hmm, now I wonder what write size pg_basebackup is actually using.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company