Re: [PG19] Promotion hangs behind a backend blocked on a stuck client

On Tue, Oct 20, 2026 at 12:20 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Tue, Oct 20, 2026 at 2:47 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Sun, Oct 18, 2026 at 5:14 PM shihao zhong <zhong950419(at)gmail(dot)com> wrote:
> >
> > TBH I'm still unsure that the 0001 patch is the correct approach.
> > ProcessProcSignalBarrier() is written on the assumption that a handler
> > can raise an error. That's why it has a PG_TRY block for a later
> > retry. In v19 we have only two handlers, ProcessBarrierSmgrRelease()
> > and ProcessBarrierUpdateXLogLogicalInfo(), and neither raises an
> > error. But on master, AbsorbDataChecksumsBarrier() raises an error on
> > purpose, with a comment saying it "will be caught by the procsignal
> > machinery for a later retry". With 0001 that error would terminate the
> > session instead of being retried. So the assumption that 0001 adds is
> > already false on master.
> >
> > Other possible solutions are: (1) send the signal barrier at promotion
> > only when we really need to send it. In other words, if we know that
> > logical decoding has been enabled from the startup and no change
> > happened during the recovery, we can skip it even if wal_level =
> > replica. (2) the startup process terminates the backend who doesn't
> > absorb the barrier, like as we resolve the recovery conflicts.
> >
> > (1) would narrow further the case where we need the wait, but doesn't
> > eliminate the case completely.
> >
>
> Right, for example, If the first logical slot appeared on the primary
> shortly before promotion, replay toggled the status (false→true), the
> case counts as "changed," and the wait remains. So, the same problem
> still exists.
>
> > (2) sounds more promising to me, and we
> > already have similar machinery such as
> > ResolveRecoveryConflictWithVirtualXIDs().
> >
>
> Like ResolveRecoveryConflictWithVirtualXIDs(), we may want to wait for
> sometime (which needs GUC or a fixed time wait) before terminating the
> backend. I think this is a direction worth pursuing but I feel it is
> too late for v19 considering the amount/complexity of changes
> required. I feel for v19, we can add a Note in the docs with a
> workaround and then discuss the more robust solution for HEAD.

Thank you for the comments.

shihao zhong drafted (1) the idea above upthread[1]. There seems to be
room for improvement there and the patch would look like the attached:
it initializes the status in shared memory from the control file, so
that the processes the postmaster forks before the startup process
reads the checkpoint record don't start with a stale cache. Given the
patch and analysis[2] shared by Manu, let me summarize the issue and
solutions.

With the v6 patch, we would need to wait for the signal barrier to be
absorbed only when the startup process enables logical decoding at the
end of recovery, which is a rare case. On standbys the status is
toggled by replaying an XLOG_LOGICAL_DECODING_STATUS_CHANGE record,
and the enable case waits for all processes. In some corner cases, the
startup process enables logical decoding at the end of recovery on
behalf of a backend creating a logical slot, so we need a wait for
that case. While making this case rarer is appealing, the v6 patch
doesn't remove it.

The v6 patch also doesn't resolve the case where the replay itself
stops because a backend is stuck writing to a client. As Amit
mentioned, if that delay happens shortly before the promotion, the
promotion would get blocked. The same can be applied to recovery
conflict cases: Manu measured that such a backend is not released by a
recovery conflict signal either[2], since
ProcessClientWriteInterrupt() services only ProcDiePending, and
therefore the recovery gets blocked even with
max_standby_streaming_delay set.

So I think that the recovery (and the promotion) getting stuck due to
client write I/O is not new in v19. max_standby_streaming_delay
doesn't work against a backend stuck writing to a client, and v19 just
adds one more way to hit the same limitation. I still prefer dealing
with this issue only for master, and leave the code in v19 as it is.
For v19 I think we should add a note to the documentation, as Amit
suggested. And possibly we can backpatch the v6 patch in a later minor
release once we test the approach enough on the master.

Regards,

[1] https://www.postgresql.org/message-id/CAGRkXqR8Kfdd%3DG1Le%3DRzhRHzZBbLtPc44207bwhrPDu_C5ZMEA%40mail.gmail.com
[2] https://www.postgresql.org/message-id/179247119467.2410654.17218914322784616346%40gmail.com

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com