Re: Asynchronous Append on postgres_fdw nodes.

On Mon, Feb 1, 2021 at 12:06 PM Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com> wrote:
> On Tue, Nov 17, 2020 at 6:56 PM Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com> wrote:
> > * I haven't yet done anything about the issue on postgres_fdw's
> > handling of concurrent data fetches by multiple ForeignScan nodes
> > (below *different* Append nodes in the query) using the same
> > connection discussed in [2]. I modified the patch to just disable
> > applying this feature to problematic test cases in the postgres_fdw
> > regression tests, by a new GUC enable_async_append.
>
> A solution for the issue would be a scheduler designed to handle such
> data fetches more efficiently, but I don’t think it’s easy to create
> such a scheduler. Rather than doing so, I'd like to propose to allow
> FDWs to disable async execution of them in problematic cases by
> themselves during executor startup in the first cut. What I have in
> mind for that is:
>
> 1) For an FDW that has async-capable ForeignScan(s), we allow the FDW
> to record, for each of the async-capable and non-async-capable
> ForeignScan(s), the information on a connection to be used for the
> ForeignScan into EState during BeginForeignScan().
>
> 2) After doing ExecProcNode() to each SubPlan and the main query tree
> in InitPlan(), we give the FDW a chance to a) reconsider, for each of
> the async-capable ForeignScan(s), whether the ForeignScan can be
> executed asynchronously as planned, based on the information stored
> into EState in #1, and then b) disable async execution of the
> ForeignScan if not.

s/ExecProcNode()/ExecInitNode()/. Sorry for that. I’ll post an
updated patch for this in a few days.

Best regards,
Etsuro Fujita