Re: [PG19] Promotion hangs behind a backend blocked on a stuck client

Hi Shihao,

Thanks for testing it again.
v3 is attached, on today's master.

On Thu, Oct 22, 2026 at 9:12 PM shihao zhong
<zhong950419(at)gmail(dot)com> wrote:
> One case is still stuck. A backend can block while it sends an error.
> It holds interrupts then, so SIGTERM does nothing. v2 still logs
> "terminating process". That can confuse users.

Reproduced with SELECT repeat('x', 30000000)::int from a client that
does not read: the backend sits in EmitErrorReport() -> ... ->
secure_write() with InterruptHoldoffCount = 1 and wait event
ClientWrite. After pg_terminate_backend() it was still there 10 s
later, 3/3 on master and on v2, and v2 logged "terminating process".

I found no clean way to tell from another process that interrupts are
held: InterruptHoldoffCount is process-local, and publishing it would
add a shared memory store to every HOLD/RESUME_INTERRUPTS(). SIGQUIT
and SIGKILL do release it, but cause a crash restart. So v3 does not
release it; it stays stuck as on master. What changes is the log. v3
says "requesting termination of process N ...", and if N is still
there at the next wakeup, "process N has not exited 5005 ms after its
termination was requested", with a DETAIL saying that it is blocked
sending to its client and will exit once the client reads or the
connection is closed. DROP DATABASE keeps waiting, and the backend
exits within milliseconds once the client reads or goes away. On TCP,
tcp_user_timeout = 5s also released it, after 5.18 s, with no
restart. The case is in 061 now.

> For the large value case, maybe add a send counter in PGPROC. If the
> counter moves during the timeout, the client is slow but alive. I
> have not tried this.

It works. v3 adds clientSendCount to PGPROC, and
internal_flush_buffer() bumps it after each completed send (a load and
a store by the owner, no lock). The waiter samples all the counts when
it starts sleeping, and terminates a ClientWrite straggler only once it
has gone procsignal_barrier_timeout without completing a send.

With 100 MB to a client reading at 2 MB/s and the default 30 s, the
session now survives and DROP waits for the value: 51.26 s (master
50.97 s; v2 terminated it at 30.07 s). The stuck cases still clear
after 2 s: I redid the coverage matrix from my mail to Amit with v3,
and no cell changes and no control was terminated. One correction to
that matrix: the slow reader control is bimodal, DROP finishing in
about 7 ms or about 0.9 s depending on where the socket is, on master
as well; it is never terminated.

A client that reads slowly enough does not move the count either: a
send only completes once the kernel has freed a sizable part of the
socket's send buffer. With a 5 s timeout, clients reading 1 byte every
3 s, 1 kB/s and 16 kB/s were terminated at the timeout, while one at
256 kB/s kept the count moving. So a live but very slow client, for
example on a poor mobile link, can still be terminated, as with v2.
Where that matters, procsignal_barrier_timeout can be raised, or set
to 0 to disable this.

In the other direction, a client reading just enough could hold the
barrier forever, so v3 also terminates a session that is still sending
once the whole wait reaches 10 times procsignal_barrier_timeout (5 min
by default). At 256 kB/s with a 5 s timeout it was terminated at
50.0 s. The factor is arbitrary; I can change it or make it a separate
setting.

The counter costs nothing measurable: pgbench -M prepared, 16 runs per
build in alternating order, v3/master was 1.007 for -S, 0.999 for one
100 kB value and 0.995 for 1000 rows, each with a 95% CI around 1.

The commit message now has Suggested-by for you.

Regards,
Manu