Re: pg_dump additional options for performance

On Tue, 26 Feb 2008, Simon Riggs wrote:

> Splitting up the dump is the enabler for splitting up the load.

While the pg_dump split train seems to be leaving the station, I feel
compelled to point out that focus does nothing to help people who are
bulk-loading data that came from somewhere else. If my data is already in
PostgreSQL, and I'm doing a dump/load, I can usually split the data easily
enough with existing tools to handle that right now via COPY (SELECT...)
TO. Some tools within pg_dump would be nice, but I don't need them that
much. It's gigantic files that came from some other DB I don't even have
access to that I struggle with loading efficiently.

The work Dimitri is doing is wandering in that direction and that may be
enough. I note that something that addresses loading big files regardless
of source could also work on PostgreSQL dumps, while a pg_dump focused
effort helps nothing but that specific workflow. I wonder if doing too
much work on the pg_dump path is the best use of someone's time when the
more general case will need to be addressed one day anyway.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD