Re: Let physical WAL senders restore missing segments from the archive

On Thu, 22 Oct 2026 at 21:43, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>
> Hi hackers,
>
> A standby can fall behind the WAL retained by its upstream while the
> missing segments are still available in the archive. We would like to
> keep that history in the archive, not retain all of it locally for a
> replication slot. Giving the receiver its own restore_command is not
> always an option either: we may be able to grant replication access
> without giving it access to the archive. Could the sender provide the
> missing WAL over the existing replication connection instead?
>
> Fujii proposed this in 2010 [0], and Japin posted a patch in 2025 [1].
> I would like to revisit it with the archive access restriction above as
> the use case, and with private copies instead of restoring into the
> normal WAL namespace.
>
> Attached is a prototype. With wal_sender_restore = on, a physical sender
> that cannot open a segment in pg_wal tries restore_command before
> failing. This works on a primary or a [cascading] standby.
>
> The temporary storage and cleanup questions came up when discussing
> reviving lost replication slots [2]. Here each sender keeps at most one
> restored segment in its own pg_wal/xlogtemp.*.walsender directory.
>
> Alexander pointed out the cost of repeated failed restores on the
> primary [3]. A request arriving within one second of a failed restore
> waits for the remainder of that interval.
>
> The existing timeline and flush/replay limits still apply. This does not
> enable logical decoding from the archive or revive an invalidated slot.
> It can serve a receiver without a slot, or with a usable physical slot.
> Joao's proposal to revalidate physical slots [4] addresses that separate
> part of the problem.
>
> The prototype runs restore_command synchronously. A slow fetch prevents
> keepalives and feedback processing, can cause receiver timeouts, and can
> delay shutdown. An interruptible implementation would need to manage the
> child command while servicing the replication connection.
>
> What do you think?
>
>
> Best regards, Andrey Borodin.
>
> [0] https://www.postgresql.org/message-id/AANLkTinN=xsPOoaXzVFSp1OkfMDAB1f_d-F91xjEZDV8@mail.gmail.com
> [1] https://www.postgresql.org/message-id/ME0P300MB0445C80039D6534452A87539B657A@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
> [2] https://www.postgresql.org/message-id/CALj2ACVUgCYVgKc42GDCFZo_=xjYyDtHXBhzyhT3s4UqAzLK4Q@mail.gmail.com
> [3] https://www.postgresql.org/message-id/CAFh8B=nH=41scx3F_EAh2H5O1w-nj8b7uCMp_0z4p4wsv2tFDA@mail.gmail.com
> [4] https://commitfest.postgresql.org/patch/6865/

Additional code review of 0001:

> /* Only physical streaming may fall back to the archive. */
> if (wal_sender_restore &&
> state->routine.segment_close == WalSndSegmentClose &&
> WalSndRestoreSegment(state, xlogfname))
> return;

This looks too vague. We are checking for func pointers - that's not
good coding at all.

> /*
> * Do not use rmtree(): this can run from a memory context reset
> * callback, where its allocations could raise another error.
> */
> snprintf(path, sizeof(path), "%s/RECOVERYXLOG", restoreDir);

I'm somewhat worried about the fact that with the 0001 string constant
"RECOVERYXLOG" is typed in core code twice.

I also didn't get why TAP test exercise cascading walsenders. 0001
itself doesn't add anything new for cascading replication, does it?
Off-list I proposed requesting WAL from replication upstream, to serve
it downstream, in case the wal file is missing locally, but this was
not implemented here.

--
Best regards,
Kirill Reshke