Re: pg_dump custom format and pigz

From: "Marc Mamin" <M(dot)Mamin(at)intershop(dot)de>
To: "Tino Schwarze" <postgresql(at)tisc(dot)de>, <pgsql-admin(at)postgresql(dot)org>
Subject: Re: pg_dump custom format and pigz
Date: 2009-10-23 13:52:16
Message-ID: C4DAC901169B624F933534A26ED7DF31010A534A@JENMAIL01.ad.intershop.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Hello,

> You might add pigz as a post-processing step and disabling compression in pg_dump.

The problem with this solution is that it make it necessary to decompress the dump entirely before using pg_restore (or did I miss a point ?)
This is an issue as our uncompressed dumps may reach 3-500 GB...

A better way would be to have a configuration option to tell which compression tool should be used internally by pg_dump.

best regards,

Marc Mamin

-----Original Message-----
From: pgsql-admin-owner(at)postgresql(dot)org [mailto:pgsql-admin-owner(at)postgresql(dot)org] On Behalf Of Tino Schwarze
Sent: Friday, October 23, 2009 3:11 PM
To: pgsql-admin(at)postgresql(dot)org
Subject: Re: [ADMIN] pg_dump custom format and pigz

Hi Marc,

On Fri, Oct 23, 2009 at 02:48:58PM +0200, Marc Mamin wrote:

> I'm using pg_dump intensively, and until now using the plaintext
> format which allows me to pipe the output to pigz
> (http://zlib.net/pigz/) This is the fastest way I've found to generate compressed backups.
>
> (These backups are very rarely used)
>
> I would like to switch to the custom format in order to take advantage
> of the parallel restore enhancement, but this would force me to give
> pigz up, hence loosing performances with pg_dump.

> Is there a way to combine the custom format with pigz?

You might add pigz as a post-processing step and disabling compression in pg_dump.

HTH,

Tino.

--
"What we nourish flourishes." - "Was wir nähren erblüht."

www.lichtkreis-chemnitz.de
www.tisc.de

--
Sent via pgsql-admin mailing list (pgsql-admin(at)postgresql(dot)org) To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Tino Schwarze 2009-10-23 14:25:25 Re: pg_dump custom format and pigz
Previous Message Tino Schwarze 2009-10-23 13:10:42 Re: pg_dump custom format and pigz