Skip site navigation (1) Skip section navigation (2)

Re: PostgreSQL 8.4 performance tuning questions

From: Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>
To: Scott Carey <scott(at)richrelevance(dot)com>
Cc: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Matthew Wakeling <matthew(at)flymine(dot)org>, "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Re: PostgreSQL 8.4 performance tuning questions
Date: 2009-07-31 06:02:58
Message-ID: 4A728912.8080809@kaltenbrunner.cc (view raw or flat)
Thread:
Lists: pgsql-performance
Scott Carey wrote:
> 
> 
> On 7/30/09 11:24 AM, "Stefan Kaltenbrunner" <stefan(at)kaltenbrunner(dot)cc> wrote:
> 
>> Kevin Grittner wrote:
>>> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>>> "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov> writes:
>>>>> Since the dump to custom format ran longer than the full pg_dump
>>>>> piped directly to psql would have taken, the overall time to use
>>>>> this technique is clearly longer for our databases on our hardware.
>>>> Hmmm ... AFAIR there isn't a good reason for dump to custom format
>>>> to take longer than plain text dump, except for applying
>>>> compression.  Maybe -Z0 would be worth testing?  Or is the problem
>>>> that you have to write the data to a disk file rather than just
>>>> piping it?
>>> I did some checking with the DBA who normally copies these around for
>>> development and test environments.  He confirmed that when the source
>>> and target are on the same machine, a pg_dump piped to psql takes
>>> about two hours.  If he pipes across the network, it runs more like
>>> three hours.
>>>
>>> My pg_dump to custom format ran for six hours.  The single-transaction
>>> restore from that dump file took two hours, with both on the same
>>> machine.  I can confirm with benchmarks, but this guy generally knows
>>> what he's talking about (and we do create a lot of development and
>>> test databases this way).
>>>
>>> Either the compression is tripling the dump time, or there is
>>> something inefficient about how pg_dump writes to the disk.
>> seems about right - compression in pg_dump -Fc is a serious bottleneck
>> and unless can significantly speed it up or make it use of multiple
>> cores (either for the dump itself - which would be awsome - or for the
>> compression) I would recommend to not use it at all.
>>
> 
> That's not an option when a dump compressed is 200GB and uncompressed is
> 1.3TB, for example.

yeah that was not meant as "don't use compression at all" but rather as 
"use a different way to compress than what pg_dump provides internally".


Stefan

In response to

pgsql-performance by date

Next:From: Rauan MaemirovDate: 2009-07-31 06:40:36
Subject: Re: PostgreSQL 8.4 performance tuning questions
Previous:From: mukeshpDate: 2009-07-31 05:49:52
Subject: Performance Monitoring tool

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group