BUG #19640: Standby permanently stuck re-requesting old timeline after promotion, never switches to new timeline

The following bug has been logged on the website:

Bug reference: 19640
Logged by: Harshit Singh
Email address: harshit(dot)singh817775(at)gmail(dot)com
PostgreSQL version: 18.0
Operating system: rhel 10
Description:

PostgreSQL version: Reproduced on both 17.6 and 18.0 — present in the
current release, not something already fixed upstream.

Operating system: Linux x86_64 (RHEL-family, built with Red Hat gcc 14.3.1)

Description:

A standby configured with recovery_target_timeline = 'latest' (the default
under streaming-replication HA managers like Patroni) can get permanently
stuck after a timeline promotion elsewhere in the cluster: it endlessly
re-requests WAL on its own old, now-superseded timeline, is told "end of WAL
reached" by the primary each time, disconnects, and immediately reconnects
requesting the same old timeline again — never advancing to the new one.
This repeats indefinitely (observed for over an hour in one case, tight
~10-20ms reconnect loop), consuming CPU, with no error surfaced to indicate
the process needs manual intervention — patronictl/monitoring tooling on top
of it just reports the node as "starting" forever.

Steps to reproduce:

1. 3-node streaming replication cluster (repro used Patroni-managed
Postgres, but the core issue appears to be in core recovery logic, not
Patroni).
2. Node A is primary; other nodes stream from it.
3. Node A is stopped/killed (or even just a plain systemctl restart of the
current leader — no exotic failure needed). Another node is promoted
(timeline N → N+1).
4. Node A is later restarted and attempts to rejoin as a replica of the new
leader.
5. Node A's local timeline is N; the new leader is on N+1.
6. Node A's log shows, repeating forever:
LOG: started streaming WAL from primary at <LSN> on timeline N
LOG: replication terminated by primary server
DETAIL: End of WAL reached on timeline N at <LSN>.
FATAL: terminating walreceiver process due to administrator command
LOG: waiting for WAL to become available at <LSN>
— new walreceiver PID each cycle, always requesting timeline N, never
N+1.

Reproduced 5 times across different sessions/timelines (N=1 through N=4) and
both PostgreSQL 17.6 and 18.0, including once under active write load (real
WAL divergence existed, not just an idle-DB edge case), and via completely
ordinary systemctl stop/start of the current leader — not an exotic
scenario. The fact that it reproduces identically on 18.0 indicates this
isn't a regression already fixed in the latest release.

Expected behavior:

The standby should detect, via rescanLatestTimeLine(), that a newer timeline
(N+1) now exists and switch its target to follow it, per
recovery_target_timeline = 'latest' semantics.

Suspected root cause / prior art:

This looks closely related to the mechanism described by Dilip Kumar on
-hackers in "Race condition in recovery?"
(https://postgrespro.com/list/thread-id/2526828)
WaitForWALToBecomeAvailable() initializes expectedTLEs from receiveTLI
rather than recoveryTargetTLI. When rescanLatestTimeLine() finds the newest
TLE already matches recoveryTargetTLI, it concludes "nothing to change" —
but expectedTLEs is left referencing the old timeline regardless, so every
subsequent WAL request keeps using it. A patch was proposed there
(initializing from recoveryTargetTLI instead) but the thread doesn't show it
as committed, and given this is still reproducible on 18.0, it appears that
patch — or an equivalent fix — never landed.

Additional context:

A near-identical symptom was reported independently against CloudNativePG
(https://github.com/cloudnative-pg/cloudnative-pg/issues/10419) consistent
with this being a core recovery-logic issue rather than something specific
to any one HA orchestration layer.