Re: Changing the state of data checksums in a running cluster

From: Daniel Gustafsson <daniel(at)yesql(dot)se>
To: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Cc: Tomas Vondra <tomas(at)vondra(dot)me>, Andres Freund <andres(at)anarazel(dot)de>, Bernd Helmle <mailings(at)oopsware(dot)de>, Michael Paquier <michael(at)paquier(dot)xyz>, Michael Banck <mbanck(at)gmx(dot)net>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Changing the state of data checksums in a running cluster
Date: 2026-03-19 23:34:02
Message-ID: 8B86AFB8-ACBC-44FC-A1AB-A58DB541A170@yesql.se
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On 19 Mar 2026, at 14:49, Daniel Gustafsson <daniel(at)yesql(dot)se> wrote:
>> On 18 Mar 2026, at 09:06, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:

>> Well, if CREATE DATABASE computed the checksums, that would fix this too. Actually, at least in the default wal_log=true mode, doesn't it already do that? It goes through the buffer cache as usual, I presume the checksums will be computed too.
>
> That's a very good point, if the CREATE DATABASE is issued with wal_log and not
> file_copy it will mark the buffers dirty during copying which will calculate
> and set the checkpoint. If file_copy isn't allowed during inprogress-on then
> the logic around rescanning databases for new entries could be quite
> simplified.

I tried to implement this on top of a fresh rebase, the attached 0003 contains
a very lightly tested version of this simplified logic (there is a removal of a
useless debug logging as well which while unrelated happened to sneak into this
work).

It now works off a single databaselist and relies on wal_copy to handle any
created database regardless of which template is used (file_copy is not allowed
while inprogress-on). This gives a chance to do better erroring out in case of
failed databases, the code is still vulnerable to a the DROP/CREATE with the
same Oid, but that seems doable to solve.

It needs more testing, and likely has bugs, but I wanted to share it as soon as
possible to see if was along what you were thinking. Personally I think the
code a lot easier to follow with this.

--
Daniel Gustafsson

Attachment Content-Type Size
v20260319-0001-Add-proper-WAL-record-for-XLOG_CHECKPOINT_.patch application/octet-stream 3.9 KB
v20260319-0002-Online-enabling-and-disabling-of-data-chec.patch application/octet-stream 231.4 KB
v20260319-0003-A-stab-at-simplified-database-processing-l.patch application/octet-stream 10.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2026-03-19 23:58:05 Re: POC: Parallel processing of indexes in autovacuum
Previous Message Robert Haas 2026-03-19 23:11:18 Re: pg_plan_advice