Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

From: Andres Freund <andres(at)anarazel(dot)de>
To: Michael Banck <michael(dot)banck(at)credativ(dot)de>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, Michael Paquier <michael(at)paquier(dot)xyz>, PostgreSQL Development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)
Date: 2021-01-06 03:35:43
Message-ID: 20210106033543.mmjrukti5mnfavaz@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On 2021-01-04 19:11:43 +0100, Michael Banck wrote:
> Am Samstag, den 02.01.2021, 10:47 -0500 schrieb Stephen Frost:
> > * Michael Paquier (michael(at)paquier(dot)xyz) wrote:
> > > On Fri, Jan 01, 2021 at 08:34:34PM +0100, Michael Banck wrote:
> > > > I think enough people use data checksums these days that it warrants to
> > > > be moved into the "normal part", like in the attached.
> > >
> > > +1. Let's see first what others think about this change.
> >
> > I agree with this, but I'd also like to propose, again, as has been
> > discussed a few times, making it the default too.

FWIW, I am quite doubtful we're there performance-wise. Besides the WAL
logging overhead, the copy we do via PageSetChecksumCopy() shows up
quite significantly in profiles here. Together with the checksums
computation that's *halfing* write throughput on fast drives in my aio
branch.

> This looks much better from the WAL size perspective, there's now almost
> no additional WAL. However, that is because pgbench doesn't do TOAST, so
> in a real-world example it might still be quite larger. Also, the vacuum
> runtime is still 15x longer.

That's obviously an issue.

> So maybe we should switch on wal_compression if we enable data checksums
> by default.

It unfortunately also hurts other workloads. If we moved towards a saner
compression algorithm that'd perhaps not be an issue anymore...

Greetings,

Andres Freund

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2021-01-06 03:36:58 Re: [HACKERS] logical decoding of two-phase transactions
Previous Message movead.li@highgo.ca 2021-01-06 03:14:37 RE: Wrong statistics for size of XLOG_SWITCH during pg_waldump.