Re: An idea for parallelizing COPY within one backend

Dimitri Fontaine wrote:
> Of course, the backends still have to parse the input given by pgloader, which
> only pre-processes data. I'm not sure having the client prepare the data some
> more (binary format or whatever) is a wise idea, as you mentionned and wrt
> Tom's follow-up. But maybe I'm all wrong, so I'm all ears!

As far as I understand, pgloader starts N threads or processes that open
up N individual connections to the server. In that case, moving then
text->binary conversion from the backend into the loader won't give any
additional performace I'd say.

The reason that I'd love some within-one-backend solution is that I'd
allow you to utilize more than one CPU for a restore within a *single*
transaction. This is something that a client-side solution won't be able
to deliver, unless major changes to the architecture of postgres happen
first...

regards, Florian Pflug