Re: backup manifests

From: Andres Freund <andres(at)anarazel(dot)de>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: Mark Dilger <mark(dot)dilger(at)enterprisedb(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Suraj Kharage <suraj(dot)kharage(at)enterprisedb(dot)com>, tushar <tushar(dot)ahuja(at)enterprisedb(dot)com>, Rajkumar Raghuwanshi <rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com>, Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Tels <nospam-pg-abuse(at)bloodgate(dot)com>, David Steele <david(at)pgmasters(dot)net>, Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Jeevan Chalke <jeevan(dot)chalke(at)enterprisedb(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>
Subject: Re: backup manifests
Date: 2020-03-27 05:31:21
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


On 2020-03-26 15:37:11 -0400, Stephen Frost wrote:
> The argument is that adding checksums takes more time. I can understand
> that argument, though I don't really agree with it. Certainly a few
> percent really shouldn't be that big of an issue, and in many cases even
> a sha256 hash isn't going to have that dramatic of an impact on the
> actual overall time.

I don't understand how you can come to that conclusion? It doesn't take
very long to measure openssl's sha256 performance (which is pretty well
optimized). Note that we do use openssl's sha256, when compiled with
openssl support.

On my workstation, with a pretty new (but not fastest single core perf
model) intel Xeon Gold 5215, I get:

$ openssl speed sha256
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
sha256 76711.75k 172036.78k 321566.89k 399008.09k 431423.49k 433689.94k

IOW, ~430MB/s.

On my laptop, with pretty fast cores:
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
sha256 97054.91k 217188.63k 394864.13k 493441.02k 532100.44k 533441.19k

IOW, 530MB/s

530 MB/s is well within the realm of medium sized VMs.

And, as mentioned before. even if you do only half of that, you're still
going to be spending roughly half of the CPU time of sending a base

What makes you think that a few hundred MB/s is out of reach for a large
fraction of PG installations that actually keep backups?


Andres Freund

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2020-03-27 05:39:05 Re: allow online change primary_conninfo
Previous Message 曾文旌 2020-03-27 05:25:33 Re: [Proposal] Global temporary tables