Skip site navigation (1) Skip section navigation (2)

Re: Parallel pg_dump for 9.1

From: David Fetter <david(at)fetter(dot)org>
To: Joachim Wieland <joe(at)mcknight(dot)de>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>,Andrew Dunstan <andrew(dot)dunstan(at)pgexperts(dot)com>
Subject: Re: Parallel pg_dump for 9.1
Date: 2010-03-29 15:07:49
Message-ID: 20100329150749.GA5544@fetter.org (view raw or flat)
Thread:
Lists: pgsql-hackers
On Mon, Mar 29, 2010 at 04:46:48PM +0200, Joachim Wieland wrote:
> People have been talking about a parallel version of pg_dump a few
> times already. I have been working on some proof-of-concept code for
> this feature every now and then and I am planning to contribute this
> for 9.1.
> 
> There are two main issues with a parallel version of pg_dump:
> 
> The first one is that it requires a consistent snapshot among
> multiple pg_dump clients

Cloning snapshots seems like the way to fix this.  I don't know how
far this project has drifted from the PostgreSQL code, but you might
want to look here:

http://pgfoundry.org/projects/snapclone/

> and the second is that currently the output goes to a single file
> and it is unclear what to do about multiple processes writing into a
> single file.

I don't think that's a good idea.  Coming up with a directory
structure for the new parallel pg_dump seems like a much better idea.

Andrew, do you have some notes on this?

Cheers,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

In response to

pgsql-hackers by date

Next:From: Robert HaasDate: 2010-03-29 15:20:21
Subject: Re: enable_joinremoval
Previous:From: Tom LaneDate: 2010-03-29 15:01:40
Subject: Re: enable_joinremoval

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group