Re: Offline data checksum changes can cause incorrect checksum state on standbys

From: Alexander Lakhin <exclusion(at)gmail(dot)com>
To: Daniel Gustafsson <daniel(at)yesql(dot)se>
Cc: Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Offline data checksum changes can cause incorrect checksum state on standbys
Date: 2026-09-26 05:00:00
Message-ID: ffd837c5-dc22-4db1-a2d8-b623cbe9b30c@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello Daniel,

14.09.2026 16:48, Daniel Gustafsson wrote:
> Pushed to master along with the cleanups from the "Trying to break online
> checksums with LLMs" thread. Not backpatched yet, as I wanted a few builds in
> the BF first and also some input on the new open item in the other thread.

BF animal turaco (Raspberry PI, kernel 6.12.75+rpt-rpi-v8) managed to fail
a test added in b52a1c2c8:
[22:51:29.073](0.002s) ok 7 - replay starts at the switchover checkpoint
[22:51:29.087](0.014s) not ok 8 - last common checkpoint is a shutdown checkpoint
[22:51:29.088](0.001s)
[22:51:29.088](0.000s) #   Failed test 'last common checkpoint is a shutdown checkpoint'
#   at t/013_rewind.pl line 161.
[22:51:29.089](0.001s) #                   ''
#     doesn't match '(?^:CHECKPOINT_SHUTDOWN)'
[22:51:29.100](0.011s) ok 9 - rewound node keeps its own checksum state in the control file
...
Test Summary Report
-------------------
t/013_rewind.pl                      (Wstat: 256 (exited 1) Tests: 13 Failed: 1)

I've reproduced this failure with the following modification:
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -3446,2 +3446,3 @@ XLogFileInitInternal(XLogSegNo logsegno, TimeLineID logtli,
         */
+pg_usleep(100000);
        installed_segno = logsegno;

which makes all-zero 000000020000000000000005 appear inside
$node_a->data_dir . '/pg_wal':
tr -d '\000' < src/test/modules/test_checksums/tmp_check/t_013_rewind_node_a_data/pgdata/pg_wal/000000020000000000000005
| wc
      0       0       0

and then if readdir() happens to return this file first (I'm observing
this on ext4):
perl -e 'opendir(my $dh, $ARGV[0]) or die; my @d = grep { length($_) == 24 } readdir($dh); print("@d\n");' \
src/test/modules/test_checksums/tmp_check/t_013_rewind_node_a_data/pgdata/pg_wal
000000020000000000000005 000000020000000000000004 000000010000000000000002 000000020000000000000003 000000010000000000000003

pg_waldump with no explicit segment specification fails:
.../pg_waldump -p src/test/modules/test_checksums/tmp_check/t_013_rewind_node_a_data/pgdata/pg_wal -s 0/03000000
pg_waldump: error: invalid WAL segment size in WAL file "000000020000000000000005" (0 bytes)
pg_waldump: detail: The WAL segment size must be a power of two between 1 MB and 1 GB.

[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=turaco&dt=2026-09-24%2019%3A58%3A27

Best regards,
Alexander

In response to

Browse pgsql-hackers by date

  From Date Subject
Previous Message Bharath Rupireddy 2026-09-26 04:24:37 Re: parallel autovacuum: Propagate track_cost_delay_timing to parallel workers