| From: | Kasper Føns <kasper(dot)fons(at)atoms(dot)co> |
|---|---|
| To: | Gabriele Bartolini <gabriele(dot)bartolini(at)gmail(dot)com> |
| Cc: | pgsql-admin(at)lists(dot)postgresql(dot)org |
| Subject: | Re: restore_command on high-throughput cluster never switches to streaming replication |
| Date: | 2026-10-23 07:11:54 |
| Message-ID: | CANOng2jmt4mkNp-YTs0752h7LvERk5GJ8qN1ZNNUccK1+00mbw@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-admin pgsql-general |
Hi Gabriele,
Thanks for following up and citing the post. Hopefully there's a solution
to be found.
Best,
Kasper
On Thu, Oct 22, 2026 at 9:20 PM Gabriele Bartolini <
gabriele(dot)bartolini(at)gmail(dot)com> wrote:
> Hi Kasper,
>
> On Thu, 22 Oct 2026 at 20:34, Kasper Føns <kasper(dot)fons(at)cloudkitchens(dot)com>
> wrote:
>
>> Hi PostgreSQL community.
>>
>> I debugged an instance where a PostgreSQL standby would not switch to
>> streaming replication when the `restore_command` fails.
>>
>> *Expectation*
>> I expect PostgreSQL to try switching to streaming replication if the
>> `restore_command` fails.
>>
>> *What happens*
>> PostgreSQL attempts to restore the previously restored WAL segment and
>> then retries the failed segment. However, because the primary produces WAL
>> at a high rate, the WAL file now exists and PostgreSQL does not try to
>> switch to streaming replication.
>>
>> *Context*
>> Running PostgreSQL 15.7 in Kubernetes using CloudNative PostgreSQL
>> Operator.
>>
>> *Logs*
>> I configured PostgreSQL to emit DEBUG3 level logs. Newest logs first,
>> oldest last.
>>
>> got WAL segment from archive
>> executing restore command "/controller/manager wal-restore
>> --log-destination /controller/log/postgres.json
>> *000000410000A7BA00000058* pg_wal/RECOVERYXLOG"
>> got WAL segment from archive
>> executing restore command "/controller/manager wal-restore
>> --log-destination /controller/log/postgres.json
>> *000000410000A7BA00000057* pg_wal/RECOVERYXLOG"
>> could not open file "pg_wal/*000000410000A7BA00000058*": No such file or
>> directory
>> could not restore file "*000000410000A7BA00000058*" from archive: child
>> process exited with exit code 1
>> executing restore command "/controller/manager wal-restore
>> --log-destination /controller/log/postgres.json
>> *000000410000A7BA00000058* pg_wal/RECOVERYXLOG"
>> got WAL segment from archive
>> executing restore command "/controller/manager wal-restore
>> --log-destination /controller/log/postgres.json
>> *000000410000A7BA00000057* pg_wal/RECOVERYXLOG"
>>
>> Notice that when *000000410000A7BA00000058* failed, PostgreSQL asked for
>> *000000410000A7BA00000057* which it had already restored. Aftwards, it
>> asks about *000000410000A7BA00000058* once again.
>>
>> *Problem*
>> This is problematic because the standby will never switch to streaming
>> replication.
>>
>> *Workaround*
>> We can get the PostgreSQL replica to become in-sync if we change the
>> command to `/bin/false` when we are withing `wal_keep_size`.
>>
>> *Question*
>> Is this the expected behaviour?
>>
>> I expect the function `WaitForWALToBecomeAvailable` to switch to
>> streaming replication once a single `restore_command` fails. This also
>> happens when `/bin/false` is used instead.
>>
>> Any help would be greatly appreciated
>> /Kasper Føns
>>
>
> Thanks for reporting this. For your information, I cited this post here:
> https://www.postgresql.org/message-id/CA%2BVUV5r0XoJdXpM_NFcxw5_YXmcHoAWb909Fu9ApUsH16Sf6_w%40mail.gmail.com
>
> I hope we will be able to solve it somehow.
>
> Ciao,
> Gabriele
>
| From | Date | Subject | |
|---|---|---|---|
| Previous Message | Gabriele Bartolini | 2026-10-22 19:20:29 | Re: restore_command on high-throughput cluster never switches to streaming replication |
| From | Date | Subject | |
|---|---|---|---|
| Previous Message | Gabriele Bartolini | 2026-10-22 19:20:29 | Re: restore_command on high-throughput cluster never switches to streaming replication |