Re: Allowing parallel pg_restore from pipe

From: Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>, Timothy Garnett <tgarnett(at)panjiva(dot)com>, pgsql-hackers(at)postgresql(dot)org, joe(at)mcknight(dot)de
Subject: Re: Allowing parallel pg_restore from pipe
Date: 2013-04-24 20:05:19
Message-ID: 51783AFF.40803@kaltenbrunner.cc
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 04/24/2013 09:51 PM, Andrew Dunstan wrote:
>
> On 04/24/2013 03:49 PM, Andrew Dunstan wrote:
>>
>> On 04/24/2013 03:40 PM, Dimitri Fontaine wrote:
>>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>>> On 04/23/2013 07:53 PM, Timothy Garnett wrote:
>>>>> Anyways, the question is if people think this is generally useful.
>>>>> If so
>>>>> I can clean up the preferred choice a bit and rebase it off of master,
>>>>> etc.
>>> I find this idea very useful yes.
>>>
>>> Another idea would be to allow for parallel pg_dump output to somehow be
>>> piped into a parallel pg_restore. I don't know how to solve that at all,
>>> it just sound something worthy of doing too.
>>>
>>
>>
>> That's not going to work, the output from parallel pg_dump is
>> inherently multiple streams. That's why it ONLY supports directory
>> format, and not even custom format on disk, let alone a pipe.
>>
>
>
> What might make sense is something like pg_dump_restore which would have
> no intermediate storage at all, just pump the data etc from one source
> to another in parallel. But I pity the poor guy who has to write it :-)

hmm pretty sure that Joachims initial patch for parallel dump actually
had a PoC for something very similiar to that...

Stefan

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2013-04-24 20:06:57 Re: Enabling Checksums
Previous Message Andrew Dunstan 2013-04-24 19:51:52 Re: Allowing parallel pg_restore from pipe