Re: PostgreSQL 8.4 performance tuning questions

On 7/30/09 11:58 AM, "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov> wrote:

> Scott Carey <scott(at)richrelevance(dot)com> wrote:
>
>> Max GZIP compression speed with the newest Intel CPU's is something
>> like 50MB/sec (it is data dependant, obviously -- it is usually
>> closer to 30MB/sec).
>
> Applying 30MB/sec to the 70GB accounts for 40 minutes. If those
> numbers are good, there's something else at play here.

It is rather data dependant, try gzip on command line as a test on some
data. On a random tarball on my Nehalem system, I just got 23MB/sec
compression rate on an uncompressable file.
Decompression with gunzip was 145MB/sec.

On a text file that I manually created with randommly placed repeating
segments that compresses 200x to 1, compression was 115MB/sec (bytes in per
sec), and decompression (bytes out per sec) was 265MB/sec.

The array in this machine will do 800MB/sec reads/sec with 'dd' and
700MB/sec writes.

One core has no chance.

Now, what needs to be known with the pg_dump is not just how fast
compression can go (assuming its gzip) but also what the duty cycle time of
the compression is. If it is single threaded, there is all the network and
disk time to cut out of this, as well as all the CPU time that pg_dump does
without compression.

> -Kevin
>