| From: | David Steele <david(at)pgmasters(dot)net> |
|---|---|
| To: | Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
| Cc: | Jeevan Chalke <jeevan(dot)chalke(at)enterprisedb(dot)com>, vignesh C <vignesh21(at)gmail(dot)com> |
| Subject: | Re: backup manifests |
| Date: | 2019-11-19 21:34:16 |
| Message-ID: | 36be23f1-0bf2-b68b-efee-76d1eb729e14@pgmasters.net |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On 11/19/19 5:00 AM, Rushabh Lathia wrote:
>
> My colleague Suraj did testing and noticed the performance impact
> with the checksums. On further testing, he found that specifically with
> sha its more of performance impact.
We have found that SHA1 adds about 3% overhead when the backup is also
compressed (gzip -6), which is what most people want to do. This
percentage goes down even more if the backup is being transferred over a
network or to an object store such as S3.
We judged that the lower collision rate of SHA1 justified the additional
expense.
That said, making SHA256 optional seems reasonable. We decided not to
make our SHA1 checksums optional to reduce the test matrix and because
parallelism largely addressed performance concerns.
Regards,
--
-David
david(at)pgmasters(dot)net
| From | Date | Subject | |
|---|---|---|---|
| Next Message | David Fetter | 2019-11-19 21:37:26 | Re: Invisible PROMPT2 |
| Previous Message | Alexander Korotkov | 2019-11-19 21:20:47 | Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7) |