|From:||Bruce Momjian <bruce(at)momjian(dot)us>|
|To:||Thomas Munro <thomas(dot)munro(at)gmail(dot)com>|
|Cc:||pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Stephen Frost <sfrost(at)snowman(dot)net>|
|Subject:||Re: Append with naive multiplexing of FDWs|
|Views:||Raw Message | Whole Thread | Download mbox | Resend email|
On Wed, Sep 4, 2019 at 06:18:31PM +1200, Thomas Munro wrote:
> A few years back I experimented with a simple readiness API that
> would allow Append to start emitting tuples from whichever Foreign
> Scan has data available, when working with FDW-based sharding. I used
> that primarily as a way to test Andres's new WaitEventSet stuff and my
> kqueue implementation of that, but I didn't pursue it seriously
> because I knew we wanted a more ambitious async executor rewrite and
> many people had ideas about that, with schedulers capable of jumping
> all over the tree etc.
> Anyway, Stephen Frost pinged me off-list to ask about that patch, and
> asked why we don't just do this naive thing until we have something
> better. It's a very localised feature that works only between Append
> and its immediate children. The patch makes it work for postgres_fdw,
> but it should work for any FDW that can get its hands on a socket.
> Here's a quick rebase of that old POC patch, along with a demo. Since
> 2016, Parallel Append landed, but I didn't have time to think about
> how to integrate with that so I did a quick "sledgehammer" rebase that
> disables itself if parallelism is in the picture.
Yes, sharding has been waiting on parallel FDW scans. Would this work
for parallel partition scans if the partitions were FDWs?
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
|Next Message||Konstantin Knizhnik||2019-09-27 16:38:00||Re: Removing unneeded self joins|
|Previous Message||Asif Rehman||2019-09-27 16:00:01||Re: WIP/PoC for parallel backup|