Re: Assertion failure in WAL summarizer

Hi Fujii,

On Thu, Oct 22, 2026 at 10:38 AM Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> I haven't investigated this yet, but wanted to report it in advance.

I can reproduce it with your steps on master, every time once the
three conditions hold, and also on REL_19, REL_18 and REL_17 with
cassert, so it goes back to when the summarizer was added.

Before promotion the summarizer has already gone past the recovery
target. In recovery GetLatestLSN() returns the walreceiver flush
position, and the walreceiver keeps streaming after replay pauses.
With your steps on master, replay pauses at the recovery target
0/03001B08, the walreceiver has flushed to 0/03001D98, and the
summarizer has summarized up to 0/03001CB0 and is waiting for WAL at
0/03001D98.

After promotion TLI 1 ends at 0/03001B08 ("1 0/03001B08 before LSN
0/03001B08" in 00000002.history), behind the summarizer. With gdb
attached to the summarizer:

1. summarizer_read_local_xlog_page() notices TLI 2 while waiting and
sets read_upto = switchpoint: 0/03001D98 -> 0/03001B08. The comment
there expects this ("this will cause read_upto to move backwards,
because we might have been promoted before reaching the end of the
previous timeline").

2. The next read is past read_upto on a timeline that is now historic,
so end_of_wal is set and SummarizeWAL() returns
summary_end_lsn = private_data->read_upto = 0/03001B08, although it
started at 0/03001CB0.

3. WalSummarizerMain(): current_lsn = 0/03001CB0,
end_of_summary_lsn = 0/03001B08 -> the assertion at line 460.

The main loop already handles "we passed the switch point": at the top
of the loop, if current_lsn >= switch_lsn it backs up to the switch
point on the new timeline. But that path only runs when the timeline
change is seen there. When it is seen inside SummarizeWAL(), the
backwards LSN comes back as the return value instead.

Without assertions nothing bad seems to happen. On a non-cassert build
of master with the same steps, the summarizer takes 0/03001B08 as its
position, sees TLI 2 on the next iteration, switches at 0/03001B08 and
goes on. It is the same process, and you end up with the same
summaries and state as with the patch below. So as far as I can tell
this only affects assert-enabled builds, but there the summarizer
crash makes the postmaster restart the whole cluster right after the
promotion.

The attached patch makes SummarizeWAL() report no progress when
read_upto ended up before the start of the summary, so the existing
code in the main loop does the rewind on the next iteration. With it,
your steps no longer trip the assertion, and summarization resumes on
TLI 2 exactly at the target, on master, 19, 18 and 17 (the patch
applies to all four). It also adds
src/bin/pg_walsummary/t/004_promote_past_target.pl, which follows your
steps: it fails with the TRAP on unpatched master and passes with the
fix. make check, pg_walsummary and pg_combinebackup pass.

Another option is to relax the assertion and keep the current
behavior, since that is already what a non-assert build does. I went
with the clamp because then SummarizeWAL() never returns an LSN before
its start, and summarized_lsn doesn't go backwards on the old timeline
for an iteration before the rewind. No strong opinion, though.

Regards,
Manu