Re: Asynchronous MergeAppend

Hi!

On Tue, Apr 14, 2026 at 12:48 PM Alexander Pyhalov
<a(dot)pyhalov(at)postgrespro(dot)ru> wrote:
> Alexander Pyhalov писал(а) 2026-04-13 18:09:
> > Looked at it. Overall I agree that when we wait for data from one slot
> > after node initialization, we can't get data from other slots - they
> > are already either exhausted, or have already received data which is
> > not needed for now. So it seems that async machinery on later stages is
> > useless.
> >
> > Was a bit surprised that asyncresults field is not used in async merge
> > append anymore, as we save result directly in ms_slots. However, as we
> > do it only on initial stage, this seems to be OK.
> >
> > classify_matching_subplans() comment still refers to ms_valid_subplans.
> >
> > Will look at it once again tomorrow.
>
> Haven't found anything suspicious (besides redundant empty line after
> enable_async_merge_append GUC description in
> src/backend/utils/misc/guc_parameters.dat). Tested it and was a bit
> surprised that new async Merge Append version (without async machinery
> after nodeMergeAppend initialization) was about 5% faster than the old
> one with concurrent queries (-j 16) and 10-15% faster with single-thread
> load (when there was a lot of CPU capacity) in my tests (test was
> basically the same as in [1]). So, async machinery after Merge Append
> node is initialized was not only useless, but even harmful.
>
> Test results:
>
> patch version | concurrency | async_capable off | async_capable on
> old | -c 1 -j 1 | 880 tps | 1428 tps (+62%)
> old | -c 16 -j 16 | 5190 tps | 4933 tps (-5%)
> new | -c 1 -j 1 | 888 tps | 1582 tps (+78%)
> new | -c 16 -j 16 | 5020 tps | 5256 tps (+4%)

While discovering a correctness of callback_pending flag reset in
async MergeAppend, I found bug in async plain Append [1]. I've
included the fix as 0001 in the current patchset. Other changes to
patchset includes.

0003 contains some cleanups
* Unify set_append_references and set_mergeappend_references (setrefs.c)
* Merge duplicate Append/MergeAppend cases in explain.c
* Merge duplicate Append/MergeAppend cases in
planstate_tree_walker_impl (nodeFuncs.c)
0006 includes following optimizations and fixes
* Fix assertion failure on MergeAppend rescan with an in-flight async
request (same as 0001 but for MergeAppend)
* Don't use ExecProcNode() for async subplans. Despite its
effectiveness, it doesn't works correctly. When postgres_fdw subplan
is executed by ExecProcNode() it interprets the end of async batch as
end of the whole data. That effectively leads to skipping the
remaining dataset after first batch (100 rows). The test is added.
* Make ExecReScanMergeAppend() clear ms_slots. Otherwise subsequent
scans can use leftover tuples. The test is also added.
* Document that the needrequest fast-skip is a no-op for MergeAppend
(postgres_fdw.c)
* Document why create_merge_append_plan discards
mark_async_capable_plan's result (createplan.c)

Links.
1. https://www.postgresql.org/message-id/CAPpHfduMOTnV5Zj2KGJ7zanL_10QvccZHtPUaDfJvBhsh9axnQ%40mail.gmail.com

------
Regards,
Alexander Korotkov
Supabase