Re: Standby stuck after walreceiver restart just before a WAL segment boundary

Oops, keyboard slipped! Hit send too early.

On Sat, Oct 31, 2026 at 8:34 AM David Shorten <shortenda(at)gmail(dot)com> wrote:

> Hello,
>
> We've found a bug where a streaming standby gets stuck if its
> walreceiver restarts
> while the last record it has received ends 8, 16 or 24 bytes before
> the end of a WAL segment, so that the next record header reaches the
> segment boundary. This issue reproduces on current master and the code
> is the same on all supported branches.
>
> To read that header, XLogPageRead() asks WaitForWALToBecomeAvailable()
> for WAL up to the end of the header, or the end of the page if the
> header crosses it. Here that is exactly the end of the segment.
> WaitForWALToBecomeAvailable() passes that end position (RecPtr) to
> RequestXLogStreaming() as the start point, and RequestXLogStreaming()
> rounds it down to a segment start. That's a no-op here, so the new
> walreceiver starts streaming at the next segment and never fetches the
> end of the current one:
>
> LOG: walreceiver requested start point 0/05000000 on timeline 1 is
> ahead of the upstream server's flush position 0/04FFFFF8, waiting
> LOG: started streaming WAL from primary at 0/05000000 on timeline 1
> LOG: invalid record length at 0/04FFFFF8: expected at least 24, got 0
> [... repeated forever]
>
> The attached patch starts streaming at the beginning of the segment
> holding RecPtr - 1, which is the segment of the page being read, as
> c06380e976 intended when it switched to RecPtr. Nothing changes unless
> RecPtr is exactly on a segment boundary. It adds a TAP test,
> t/061_walrcv_restart_segment_boundary.pl, which pads WAL so that the
> last record ends 8 bytes before a boundary, restarts the walreceiver by
> changing primary_conninfo, and checks where streaming restarts. It
> fails on master without the fix and passes with it; I also checked gaps
> of 16 and 24 bytes by changing the test. src/test/recovery passes with
> make and meson.

+1, i have reviewed the patch, i think we can simply use
ptr = RecPtr - 1 instead of the conversions done
in WaitForWALToBecomeAvailable
, I think RequestXLogStreaming will handle those.

Regarding the TAP test, instead of padding WAL to precisely the boundary -
8 and
relying on a 5-attempt retry loop to remove unwanted WALs like
XLOG_RUNNING_XACTS
to avoid WAL between padding and the walreceiver restart, could we
construct this state directly?
We could generate enough WAL to cross the boundary on the primary, take a
basebackup,
and then truncate the standby's copy of that last WAL segment by 8 bytes
before starting the
replica. This would guarantee the exact starting LSN without the retry loop.

> I think it makes sense to backport the fix since it applies to
> REL_15_STABLE through REL_19_STABLE. REL_14_STABLE
> has the same code in xlog.c, and I can send a version for it.
>
+1

--
Thanks :)
Srinath Reddy Sadipiralla
EDB: https://www.enterprisedb.com/