Re: BUG #19640: Standby permanently stuck re-requesting old timeline after promotion, never switches to new timeline

From: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
To: harshit(dot)singh817775(at)gmail(dot)com, PostgreSQL mailing lists <pgsql-bugs(at)lists(dot)postgresql(dot)org>
Subject: Re: BUG #19640: Standby permanently stuck re-requesting old timeline after promotion, never switches to new timeline
Date: 2026-08-27 16:17:30
Message-ID: 6E6B47DF-8950-4BD4-AA4E-FC9EF4D53CBC@yandex-team.ru
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi Harshit,

On Wed, Aug 26, 2026 at 11:56 AM PG Bug reporting form wrote:
> Node A's local timeline is N; the new leader is on N+1.
> Node A's log shows, repeating forever:
> LOG: started streaming WAL from primary at <LSN> on timeline N
> DETAIL: End of WAL reached on timeline N at <LSN>.

There are several cases that can produce a symptom like this, and the
actual LSNs are important for distinguishing them.

First, Node A was the former primary. If the new primary forked from
timeline N at a position behind Node A's local WAL, Node A is already
past the switchpoint and cannot continue on N+1. It has to be rewound
or rebuilt. Could you check whether Patroni attempted pg_rewind,
whether it succeeded, and whether data checksums or wal_log_hints make
pg_rewind possible in this cluster? If Patroni did not run pg_rewind,
or if it failed, that part is probably better investigated with the
Patroni project.

For this thread, please include the complete PostgreSQL log around one
cycle, including the startup process and walreceiver messages, rather
than only the repeated lines.

The proposed receiveTLI -> recoveryTargetTLI fix from the 2021 thread
was committed as 6eb5b9ae391 [0]. It was released in PostgreSQL
9.6.23, 10.18, 11.13, 12.8, and 13.4, and has been present since 14.0.
In particular, both 17.6 and 18.0 include it. So that particular
explanation should not apply here.

There are two nearby core recovery issues, but I would consider them
only after ruling out the cases above. Recovery can currently read an
old timeline's copy of a switchpoint segment when the new timeline's
copy is not available yet [1]. That can move replay past the
switchpoint on the wrong timeline. There is also a smaller window where
startup can stop a walreceiver while it is fetching the new timeline
history file, before it enters WALRCV_WAITING.

Could you provide:

* the switchpoint from the N+1 history file;
* the real start and end LSNs from the repeated messages;
* Node A's latest checkpoint and redo positions from pg_controldata;
* the complete log from the first attempt to follow N+1;
* whether pg_rewind was attempted, and its final result.

Those values should tell us whether Node A was already divergent,
replayed past the switchpoint during this startup, or reached the
switchpoint but failed to change timelines.

Thank you!

Best regards, Andrey Borodin.

[0] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=6eb5b9ae39176a7d40003b4c2e9ca22e6b205def
[1] https://www.postgresql.org/message-id/flat/85386EF6-16B7-4D62-86BE-526A10F93825%40yandex-team.ru

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Alexander Korotkov 2026-08-27 20:24:13 Re: MERGE/SPLIT PARTITIONS issues/questions
Previous Message Pierre Forstmann 2026-08-27 15:49:50 Re: BUG #19369: Not documented that io_uring on kernel versions between 5.1 and below 5.6 does not work