Skip site navigation (1) Skip section navigation (2)

Re: pg_dump additional options for performance

From: Gregory Stark <stark(at)enterprisedb(dot)com>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Simon Riggs" <simon(at)2ndquadrant(dot)com>, "Jeff Davis" <pgsql(at)j-davis(dot)com>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_dump additional options for performance
Date: 2008-02-26 20:14:49
Message-ID: (view raw or whole thread)
Lists: pgsql-hackers
"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
>> I've not been advocating improving pg_restore, which is where the -Fc
>> tricks come in.
>> ...
>> I see you thought I meant pg_restore. I don't thinking extending
>> pg_restore in that way is of sufficiently generic use to make it
>> worthwhile. Extending psql would be worth it, since not all psql scripts
>> come from pg_dump.
> OK, the reason I didn't grasp what you are proposing is that it's insane.
> We can easily, and backwards-compatibly, improve pg_restore to do
> concurrent restores.  Trying to make psql do something like this will
> require a complete rewrite, and there is no prospect that it will work
> for any input that didn't come from (an updated version of) pg_dump
> anyway.  

I didn't read everything in the thread previously so I'm not sure if this is
what Simon had in mind. But I think one thing that could be done in parallel
even in psql scripts is index builds. That doesn't help speed up COPY but it
does speed up a case where we currently are limited by only occupying a single
cpu. And I would expect it to play well With synchronized scans too.

The "complete rewrite" in this case would be the "concurrent psql" patch I
submitted a while back. I think it's a bit of a mess right now because I was
trying to chase down some bugs with sigint handling so I've been thinking of
rewriting it.

I think this is a low-hanging fruit which would help a lot of users. The
ability to load multiple COPY dumps would be the other piece of the puzzle but
personally I'm not sure how to tackle that.

  Gregory Stark
  Ask me about EnterpriseDB's PostGIS support!

In response to


pgsql-hackers by date

Next:From: Gregory StarkDate: 2008-02-26 20:22:01
Subject: Re: Proposed changes to DTrace probe implementation
Previous:From: Neil ConwayDate: 2008-02-26 20:06:39
Subject: Re: code cleanup of timestamp code

Privacy Policy | About PostgreSQL
Copyright © 1996-2015 The PostgreSQL Global Development Group