Re: Allowing parallel pg_restore from pipe

On Wed, Apr 24, 2013 at 6:47 PM, Joachim Wieland <joe(at)mcknight(dot)de> wrote:
> On Wed, Apr 24, 2013 at 4:05 PM, Stefan Kaltenbrunner
> <stefan(at)kaltenbrunner(dot)cc> wrote:
>>
>> > What might make sense is something like pg_dump_restore which would have
>> > no intermediate storage at all, just pump the data etc from one source
>> > to another in parallel. But I pity the poor guy who has to write it :-)
>>
>> hmm pretty sure that Joachims initial patch for parallel dump actually
>> had a PoC for something very similiar to that...
>
>
> That's right, I implemented that as an own output format and named it
> "migrator" I think, which wouldn't write each stream to a file as the
> directory output format does but that instead pumps it back into a restore
> client.
>
> Actually I think the logic was even reversed, it was a parallel restore that
> got the data from internally calling pg_dump functionality instead of from
> reading files... The neat thing about this approach was that the order was
> optimized and correct, i.e. largest tables start first and dependencies get
> resolved in the right order.
>
> I could revisit that patch for 9.4 if enough people are interested.

Indeed... I've wasted hours copying databases for test environments,
when that could've been hour (singular).