Re: pg_dump and pg_restore with multiple streams does Not seem to improve overall times

From: Jan Lentfer [mailto:Jan(dot)Lentfer(at)web(dot)de]
Sent: Monday, May 04, 2015 2:03 PM
To: Igor Neyman; Mel Llaguno
Cc: pgsql-admin(at)postgresql(dot)org
Subject: RE: [ADMIN] pg_dump and pg_restore with multiple streams does Not seem to improve overall times

Am 4. Mai 2015 19:22:07 MESZ, schrieb Igor Neyman <ineyman(at)perceptron(dot)com<mailto:ineyman(at)perceptron(dot)com>>:

-----Original Message-----
From: pgsql-admin-owner(at)postgresql(dot)org<mailto:pgsql-admin-owner(at)postgresql(dot)org> [mailto:pgsql-admin-owner(at)postgresql(dot)org] On Behalf Of Jan Lentfer
Sent: Monday, May 04, 2015 12:42 PM
To: Mel Llaguno
Cc: pgsql-admin(at)postgresql(dot)org<mailto:pgsql-admin(at)postgresql(dot)org>
Subject: Re: [ADMIN] pg_dump and pg_restore with multiple streams does Not seem to improve overall times

I don't know about parallel pg_dump as we use -Fc and pg_dump can't do that in parallel (afaik). For dumping I have wrapped pg_dump in a shell script to dump several databases in parallel.
But for pg_restore -j option does make a big difference, at least when you have a lot of larger tables and indexes.

Regards,

Jan

Combination of -Fc and -j definitely works in pg_dump. And there is nothing in the docs that states otherwise.

Regards,
Igor Neyman

Well, the manual says

You can only use this option with the directory output format because this is the only output format where multiple processes can write their data at the same time.

For the -j Option

Regards

Jan

Sorry, was talking (for some reason?) about parallel restore, which was available long before parallel dump.

Igor Neyman