pgsql: Re-register LSN waiters after stale wakeups

Re-register LSN waiters after stale wakeups

WaitLSNWakeup() removes a waiter from the heap before setting its latch.
If the position that caused the wakeup moves backwards before the waiter
rechecks it, as can happen when WAL streaming restarts, the waiter may
sleep again while no longer registered. Subsequent WAL progress then
cannot wake it.

When an unmet waiter finds that it is no longer in the heap, add it back
and restart the loop. Rereading the position after registration also
prevents missing an advance between the previous read and the re-add.

Process interrupts before re-registering rather than after, so that a
wakeup which goes stale again cannot postpone cancellation, however often
it repeats. As a side effect, a pending cancel now wins over an expired
timeout, which previously reported a timeout instead.

Add deterministic TAP coverage that simulates a stale standby_write
wakeup without advancing the actual write or replay positions.

Author: Xuneng Zhou <xunengzhou(at)gmail(dot)com>
Reviewed-by: Alexander Korotkov <aekorotkov(at)gmail(dot)com>
Discussion: https://postgr.es/m/CABPTF7UtW_cAa%3DQh4RDfKiUqu3pJJE22ai9tbWJVERbeRyssLw%40mail.gmail.com
Backpatch-through: 19

Branch
------
REL_19_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/0ec36f4ba325062882eb7a36b8f815e85ff8c123

Modified Files
--------------
src/backend/access/transam/xlogwait.c | 37 +++++++-
src/test/modules/Makefile | 1 +
src/test/modules/meson.build | 1 +
src/test/modules/test_wait_lsn/Makefile | 21 +++++
src/test/modules/test_wait_lsn/meson.build | 22 +++++
.../modules/test_wait_lsn/test_wait_lsn--1.0.sql | 14 +++
src/test/modules/test_wait_lsn/test_wait_lsn.c | 99 ++++++++++++++++++++++
.../modules/test_wait_lsn/test_wait_lsn.control | 4 +
src/test/recovery/Makefile | 3 +-
src/test/recovery/t/049_wait_for_lsn.pl | 85 +++++++++++++++++++
10 files changed, 284 insertions(+), 3 deletions(-)