Skip site navigation (1) Skip section navigation (2)

Re: Improve compression speeds in pg_lzcompress.c

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Merlin Moncure <mmoncure(at)gmail(dot)com>
Cc: Takeshi Yamamuro <yamamuro(dot)takeshi(at)lab(dot)ntt(dot)co(dot)jp>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Improve compression speeds in pg_lzcompress.c
Date: 2013-01-07 20:41:00
Message-ID: 16576.1357591260@sss.pgh.pa.us (view raw or flat)
Thread:
Lists: pgsql-hackers
Merlin Moncure <mmoncure(at)gmail(dot)com> writes:
> On Mon, Jan 7, 2013 at 10:16 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Takeshi Yamamuro <yamamuro(dot)takeshi(at)lab(dot)ntt(dot)co(dot)jp> writes:
>>> The attached is a patch to improve compression speeds with loss of
>>> compression ratios in backend/utils/adt/pg_lzcompress.c.

>> Why would that be a good tradeoff to make?  Larger stored values require
>> more I/O, which is likely to swamp any CPU savings in the compression
>> step.  Not to mention that a value once written may be read many times,
>> so the extra I/O cost could be multiplied many times over later on.

> I disagree.  pg compression is so awful it's almost never a net win.
> I turn it off.

One report doesn't make it useless, but even if it is so on your data,
why would making it even less effective be a win?

>> Another thing to keep in mind is that the compression area in general
>> is a minefield of patents.  We're fairly confident that pg_lzcompress
>> as-is doesn't fall foul of any, but any significant change there would
>> probably require more research.

> A minefield of *expired* patents.  Fast lz based compression is used
> all over the place -- for example by the lucene.

The patents that had to be dodged for original LZ compression are gone,
true, but what's your evidence for saying that newer versions don't have
newer patents?

			regards, tom lane


In response to

Responses

pgsql-hackers by date

Next:From: Alvaro HerreraDate: 2013-01-07 20:43:28
Subject: Re: ALTER command reworks
Previous:From: Merlin MoncureDate: 2013-01-07 20:18:57
Subject: Re: Improve compression speeds in pg_lzcompress.c

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group