Re: Offline data checksum changes can cause incorrect checksum state on standbys

From: Daniel Gustafsson <daniel(at)yesql(dot)se>
To: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Cc: Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Offline data checksum changes can cause incorrect checksum state on standbys
Date: 2026-09-08 09:12:40
Message-ID: CF434E09-1A19-4A18-9A59-7CD82D31BBEF@yesql.se
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On 8 Sep 2026, at 10:58, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>
> On 08/09/2026 11:17, Daniel Gustafsson wrote:
>>> On 7 Sep 2026, at 17:57, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>>> 3: Primary on, replica off
>>>
>>> You end up in this situation, if you turn on run pg_checksums to turn on checksums in primary. You stay that you really really shouldn't stay for long in this state, but why? What's the harm?
>>>
>>> One harm is that it's confusing, but if we have to deal with it anyway, why is it so bad?
>> Tools like pg_rewind etc rely on the fact that nodes are equal in checksum
>> state. What if the replica is promoted? I'm personally unconvinced that there
>> is a good usecase for per-node settings, but I've spent X years thinking about
>> checksums being replicated so I am clearly biased.
>> Considering how complicated it was to get replicated states right, if we want
>> to make it per-node I think we need to go back to the drawing board and
>> re-think properly rather than settle for that it seems to work. (Not that I
>> think that's what you're advocating, I just expect getting it work will be
>> complicated.)
>
> Sure, I don't see any point in this setup either. It's just something that you can end up with. But as long as you can end up with it, we need to deal with it gracefully.

Agreed, that's the gist of the patch in this thread. Identify, issue WARNINGs
on the secondary and make sure tools like pg_rewind don't break things. The
cluster will continue to operate and the state can be unified.

> If the replica is promoted, that seems fine. Checksums will be off.
>
> In principle, I think pg_rewind would still work as long as wal_log_hints=on. But I don't think we need to cater for that, erroring out is fine.
>
>>> 4: Primary off, replica on
>>>
>>> Is this possible? Does it make sense?
>> This is possible using pg_checksums, but like the inverse case I don't think it
>> makes sense to use different settings across replication.
>
> How about we forbid this completely? Forbid running pg_checksums on replica, unless the primary already has checksums on.
>
> I guess that would make it impossible to run pg_checksums concurrently in the primary and the standby. You'd have to run pg_checksums on primary first, wait for it to finish, and only then launch it in the standby.

Since pg_checksums work on offline clusters in isolation it (currently) doesn't
know about other nodes or replication at all.

> Maybe print a warning if you run pg_checksums in a replica:
>
> WARNING: unless you run pg_checksums on the primary at the same time, the checksums will be immediately disabled again after startup

Regardless of the fate of online checksums in 19 I think it would be good to
issue a warning in pg_checksums that all nodes need to be modified.

--
Daniel Gustafsson

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Chao Li 2026-09-08 09:14:56 Fix inherited constraint loss in ALTER COLUMN SET EXPRESSION
Previous Message Grigorev Jurij 2026-09-08 09:06:36 DSA_ALLOC_NO_OOM vs dsm_create ERROR leaving a half-initialized pgstats hash entry