GZIP of pre-zipped output

From: Dave Crooke <dcrooke(at)gmail(dot)com>
To: David Newall <postgresql(at)davidnewall(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org, robertmhaas(at)gmail(dot)com, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Craig Ringer <craig(at)postnewspapers(dot)com(dot)au>
Subject: GZIP of pre-zipped output
Date: 2010-03-21 17:04:00
Message-ID: ca24673e1003211004l68237f72r101eb04082f8e288@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

If you are really so desparate to save a couple of GB that you are resorting
to -Z9 then I'd suggest using bzip2 instead.

bzip is designed for things like installer images where there will be
massive amounts of downloads, so it uses a ton of cpu during compression,
but usually less than -Z9 and makes a better result.

Cheers
Dave

On Mar 21, 2010 10:50 AM, "David Newall" <postgresql(at)davidnewall(dot)com> wrote:

Tom Lane wrote:
>
> I would bet that the reason for the slow throughput is that gzip
> is fruitlessl...
Indeed, I didn't expect much reduction in size, but I also didn't expect a
four-order of magnitude increase in run-time (i.e. output at 10MB/second
going down to 500KB/second), particularly as my estimate was based on
gzipping a previously gzipped file. I think it's probably pathological
data, as it were. Might even be of interest to gzip's maintainers.

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Bob Lunney 2010-03-21 18:03:30 Re: pg_dump far too slow
Previous Message David Newall 2010-03-21 15:50:34 Re: pg_dump far too slow