Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)

From: "Robert Haas" <robertmhaas(at)gmail(dot)com>
To: "Douglas McNaught" <doug(at)mcnaught(dot)org>
Cc: "Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>, "Alvaro Herrera" <alvherre(at)commandprompt(dot)com>, lar(at)quicklz(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)
Date: 2009-01-05 18:04:14
Message-ID: 603c8f070901051004w35b23e2j7c1c9d908faf0480@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> Are
> we willing to be dropped from Debian and possibly Red Hat if this is
> the case?

No. I frankly think this discussion is a dead end.

The whole thing got started because Alex Hunsaker pointed out that his
database got a lot bigger because we disabled compression on columns >
1MB. It seems like the obvious thing to do is turn it back on again.
The only objection to that is that it will hurt performance,
especially on substring operations. That lead to a discussion of
alternative compression algorithms, which is only relevant if we
believe that there are people out there who want to do substring
extractions on huge columns AND want those columns to be compressed.
At least on this thread, we have zero requests for that feature
combination.

What we do have is a suggestion from several people that the database
shouldn't be in the business of compressing data AT ALL. If we want
to implement that suggestion, then we could change the default column
storage type.

Regardless of whether we do that or not, no one has offered any
justification of the arbitrary decision not to compress columns >1MB,
and at least one person (Peter) has suggested that it is exactly
backwards. I think he's right, and this part should be backed out.
That will leave us back in the sensible place where people who want
compression can get it (which is currently false) and people who don't
want it can get rid of it (which has always been true). If there is a
demand for alternative compression algorithms, then someone can submit
a patch for that for 8.5.

...Robert

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2009-01-05 18:07:55 Re: version() output vs. 32/64 bits
Previous Message Stefan Kaltenbrunner 2009-01-05 17:43:23 Re: QuickLZ compression algorithm (Re: Inclusion in the PostgreSQL backend for toasting rows)