Re: Improving compressibility of WAL files

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Greg Smith" <gsmith(at)gregsmith(dot)com>, "Hannu Krosing" <hannu(at)krosing(dot)net>
Cc: "Aidan Van Dyk" <aidan(at)highrise(dot)ca>, "Kyle Cordes" <kyle(at)kylecordes(dot)com>, "Bruce Momjian" <bruce(at)momjian(dot)us>, "PostgreSQL-development" <pgsql-hackers(at)postgreSQL(dot)org>
Subject: Re: Improving compressibility of WAL files
Date: 2009-01-09 15:58:21
Message-ID: 49671FBD.EE98.0025.0@wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

>>> Greg Smith <gsmith(at)gregsmith(dot)com> wrote:
> I thought at one point that the direction this was going toward was
to
> provide the size of the WAL file as a parameter you can use in the
> archive_command: %p provides the path, %f the file name, and now %l
the
> length. That makes an example archive command something like:
>
> head -c "%l" "%p" | gzip > /mnt/server/archivedir/"%f"

Hard to beat for performance. I thought there was some technical
snag.

> Expanding it back to always be 16MB on the other side might require
some
> trivial script, can't think of a standard UNIX tool suitable for that
but
> it's easy enough to write.

Untested, but it seems like something close to this would work:

cat $p $( dd if=/dev/null blocks=1 ibs=$(( (16 * 1024 * 1024) - $(stat
-c%s $p) )) )

-Kevin

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2009-01-09 16:02:15 Re: Improving compressibility of WAL files
Previous Message Mohamed 2009-01-09 15:44:17 Adding Arabic dictionary for TSearch2.. to_tsvector('arabic'...) doesn't work..

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2009-01-09 16:02:15 Re: Improving compressibility of WAL files
Previous Message Hiroshi Saito 2009-01-09 15:57:08 Re: Solve a problem of LC_TIME of windows.