Re: CREATE TABLE with parallel workers, 10.0?

Andres,

* Andres Freund (andres(at)anarazel(dot)de) wrote:
> On 2017-02-15 20:28:43 -0500, Stephen Frost wrote:
> > * Andres Freund (andres(at)anarazel(dot)de) wrote:
> > > On February 15, 2017 5:20:20 PM PST, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> > > >In many cases, I expect this would work just as well, if not better,
> > > >than trying to actually do writes in parallel.
> > >
> > > Why? IPCing tuples around is quite expensive. Or do you mean because it'll be more suitable because of the possible plans?
> >
> > Because I've seen some serious problems when trying to have multiple
> > processes writing into the same relation due to the relation extension
> > lock, cases where it was much faster to have each process write into its
> > own table. Admittedly, we've improved things there, so perhaps this isn't
> > an issue any longer, but we also don't yet really know what the locking
> > implementation looks like yet for having multiple parallel workers
> > writing into the same relation, so it may be that sending a few records
> > back to the leader is cheaper than working out the locking to allow
> > parallel workers to write to the same relation, or at least not any more
> > expensive.
>
> I quite seriously doubt that you will get enough rows to the master via
> tuplequeues that it'll be faster than inserting on the workers, eve
> nwith such scalability problems present. Even before the 9.6
> improvements, and even more so after.

Perhaps, but until we've got a system worked out for having the workers
do the writes, we aren't getting anything. Being able to have the
leader do the writes based on the tuples fed back from the workers is
clearly better than nothing.

Thanks!

Stephen