Re: Improving compressibility of WAL files

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Simon Riggs" <simon(at)2ndQuadrant(dot)com>, "Greg Smith" <gsmith(at)gregsmith(dot)com>
Cc: "Aidan Van Dyk" <aidan(at)highrise(dot)ca>, "Kyle Cordes" <kyle(at)kylecordes(dot)com>, "Bruce Momjian" <bruce(at)momjian(dot)us>, "PostgreSQL-development" <pgsql-hackers(at)postgresql(dot)org>, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject: Re: Improving compressibility of WAL files
Date: 2009-01-09 21:31:20
Message-ID: 49676DC8.EE98.0025.0@wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

>>> Greg Smith <gsmith(at)gregsmith(dot)com> wrote:
> The main use case I'm concerned about losing support for is:
>
> 1) Two systems connected by a WAN with significant transmit latency
> 2) The secondary system runs a warm standby aimed at disaster
recovery
> 3) Business requirements want the standby to never be more than (say)
5
> minutes behind the primary, presuming the WAN is up
> 4) WAN traffic is "expensive" (money==bandwidth, one of the two is
scarce)
>
> This seems a pretty common scenario in my experience. Right now,
this
> case is served quite well like this:
>
> -archive_timeout='5 minutes'
> -[pglesslog|pg_clearxlogtail] | gzip | rsync

You've come pretty close to describing our environment, other than
having 72 primaries each using rsync to push the WAL files to another
server at the same site while a server at the central site uses rsync
to pull them back. We don't run warm standby on the backup server at
the site of origin, and don't want to have to do so.

It is critically important that the flow of xlog data never hold up
the primary databases, and that failure to copy xlog to either of the
targets not interfere with copying to the other. (We have WAN
failures surprising often, sometimes for days at a time, and the
backup server on-site is in the same rack of the same cabinet as the
database server.)

Compression of xlog data is important not only for WAN transmission,
but for storage space. We keep two weeks of WAL files to allow
recovery from either of the last two weekly backups, and we archive
the first weekly backup of each month, with the WAL files needed for
recovery, for one year.

So it appears we care about somewhat similar issues.

-Kevin

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Ivan Sergio Borgonovo 2009-01-09 23:29:30 refactoring: changing tables names
Previous Message Greg Smith 2009-01-09 20:58:14 Re: Improving compressibility of WAL files

Browse pgsql-hackers by date

  From Date Subject
Next Message Hiroshi Inoue 2009-01-09 21:45:46 Re: Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Previous Message Tom Lane 2009-01-09 21:30:49 Re: Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.