Re: PostgreSQL 8.4 performance tuning questions

On Tue, Aug 4, 2009 at 4:40 PM, Tom Lane<tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Scott Carey <scott(at)richrelevance(dot)com> writes:
>> There are a handful of other compression algorithms very similar to LZO in
>> performance / compression level under various licenses.
>> LZO is just the best known and most widely used.
>
> And after we get done with the license question, we need to ask about
> patents.  The compression area is just a minefield of patents.  gzip is
> known to avoid all older patents (and would be pretty solid prior art
> against newer ones).  I'm far less confident about lesser-known systems.

I did a little bit of research. LZO and friends are variants of LZW.
The main LZW patent died in 2003, and AFAIK there has been no patent
enforcement cases brought against LZO or it's cousins (LZO dates to
1996). OK, I'm no attorney, etc, but the internet seems to believe
that the algorithms are patent free. LZO is quite widely used, in
both open source and some relatively high profile commercial projects.

I downloaded the libraries and did some tests.
2.5 G sql dump:

compression time:
zlib: 4m 1s
lzo: 17s
fastlz: 28.8s
liblzf: 26.7s

compression size:
zlib: 609M 75%
lzo: 948M 62%
fastlz: 936M 62.5%
liblzf: 916M 63.5%

A couple of quick notes: liblzf produces (possibly) architecture
dependent archives according to its header, and fastlz is not declared
'stable' according to its website.

merlin