pgsql: Drain pending asynchronous requests during ExecReScanAppend.

Drain pending asynchronous requests during ExecReScanAppend.

The logic for asynchronous Append assumes that pending requests made for
subplans of an Append are drained during ExecReScanAppend. To ensure
that, commit 9e283fc85 modified postgresReScanForeignScan to drain such
a request if any, but failed to take into account that if such a request
was made for a subplan that is re-scanned with parameter changes or
pruned in the next round by runtime pruning, the postgres_fdw callback
function is called after ExecReScanAppend or never called, respectively.
This would cause such a request to remain even after ExecReScanAppend,
leading to incorrect results, an infinite loop, or an assertion failure.

To fix, modify ExecReScanAppend to, for each of the pending requests,
give the FDW a chance to drain that request using the existing
ForeignAsyncConfigureWait/ForeignAsyncNotify callback functions. This
makes the change made to postgresReScanForeignScan useless, so remove it
as well.

Back-patch to v14 where asynchronous Append was added.

Reported-by: Alexander Korotkov <aekorotkov(at)gmail(dot)com>
Co-authored-by: Alexander Korotkov <aekorotkov(at)gmail(dot)com>
Co-authored-by: Gleb Kashkin <g(dot)kashkin(at)postgrespro(dot)ru>
Co-authored-by: Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>
Reviewed-by: Alexander Pyhalov <a(dot)pyhalov(at)postgrespro(dot)ru>
Reviewed-by: Gleb Kashkin <g(dot)kashkin(at)postgrespro(dot)ru>
Discussion: https://postgr.es/m/CAPpHfduMOTnV5Zj2KGJ7zanL_10QvccZHtPUaDfJvBhsh9axnQ%40mail.gmail.com
Backpatch-through: 14

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/f57df206814d2f15a95cd452931b464a62740165

Modified Files
--------------
contrib/postgres_fdw/expected/postgres_fdw.out | 70 ++++++++++++++++++-
contrib/postgres_fdw/postgres_fdw.c | 15 ++--
contrib/postgres_fdw/sql/postgres_fdw.sql | 14 +++-
src/backend/executor/nodeAppend.c | 97 +++++++++++++++++++++-----
4 files changed, 163 insertions(+), 33 deletions(-)