TOAST compression

From: Neil Conway <neilc(at)samurai(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: TOAST compression
Date: 2006-02-26 02:39:34
Message-ID: 1140921574.8830.105.camel@localhost.localdomain
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

toast_compress_datum() considers compression to be "successful" if the
compressed version of the datum is smaller than the uncompressed
version. I think this is overly generous: if compression reduces the
size of the datum by, say, 0.01%, it is likely a net loss to use the
compressed version of the datum since we'll need to pay for LZ
decompression every time that we de-TOAST it. This situation can occur
frequently when storing "mostly-uncompressible" data (compressed images,
encrypted data, etc.) -- some parts of the data will compress well (e.g.
metadata), but the vast majority will not.

It's true that LZ decompression is fast, so we should probably use the
compressed version of the datum unless the reduction in size is very
small. I'm not sure precisely what that threshold should be, however.

Comments?

-Neil

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2006-02-26 03:11:10 Re: TOAST compression
Previous Message James William Pye 2006-02-26 01:53:35 Re: Pl/Python -- current maintainer?