Re: Switching XLog source from archive to streaming when primary available

From: Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
To: Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>
Cc: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>, Nathan Bossart <nathandbossart(at)gmail(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, Ian Lawrence Barwick <barwick(at)gmail(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, Cary Huang <cary(dot)huang(at)highgo(dot)ca>, SATYANARAYANA NARLAPURAM <satyanarlapuram(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Switching XLog source from archive to streaming when primary available
Date: 2026-11-19 10:59:22
Message-ID: CAN55FZ0un_7O9qAwZcHPYDMfAp6zQzazR1qjPuww_qLz=bJhcQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On Wed, 23 Sept 2026 at 12:51, Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com> wrote:
>
> I have slightly simplified the patch, while preserving its original semantics.
>
> The patch introduces a timeout-based server parameter,
> "streaming_replication_retry_interval". When a standby has been
> reading WAL from the archive for the configured interval, it attempts
> to switch to streaming replication. Before doing so, it first ensures
> that all WAL already available under the pg_wal directory has been
> consumed. If streaming starts successfully, it continues using the
> streaming source. Otherwise, it falls back to archive recovery.
>
> Please have a look and let me know your thoughts.

I want to talk about another approach. Could the standby report its
progress to the primary while it is replaying WAL from the archive,
without having to switch to streaming first?

The idea is to keep walreceiver running in a feedback-only mode while
recovery reads from the archive or local pg_wal. It would connect to
the primary and send periodic updates. If the primary were
unreachable, archive recovery could continue while walreceiver
retries.

The main benefits I can think of are:

1. If the streaming is preferred and available, the standby can switch
directly to streaming [*]. Feedback-only mode is useful when restoring
from the archive is faster, or when the primary is reachable but
streaming no longer works.

2. WAL position updates could advance the slot’s restart_lsn, so the
primary would not retain WAL for that slot because streaming has not
resumed. This helps synchronized_standby_slots as well.

3. When hot_standby_feedback is enabled, the same connection could
report which row versions standby queries still need. This could help
prevent query cancellations.

[*] I think this could work alongside v25: periodic retries decide
when to switch back to streaming, while feedback-only mode lets the
standby report progress until that switch succeeds.

At least one trade-off is that the proposed approach is more
complicated than v25.

What do you think? Do you think this idea is worth to pursue?

--
Regards,
Nazir Bilal Yavuz
Microsoft

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2026-11-19 11:43:41 Re: Avoid rescanning prefixes in appendStringLiteralDQ()
Previous Message Fujii Masao 2026-11-19 10:59:03 Re: psql: avoid over-reading unterminated prompt escapes