Skip site navigation (1) Skip section navigation (2)

TOAST compression

From: Neil Conway <neilc(at)samurai(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: TOAST compression
Date: 2006-02-26 02:39:34
Message-ID: 1140921574.8830.105.camel@localhost.localdomain (view raw or flat)
Thread:
Lists: pgsql-hackers
toast_compress_datum() considers compression to be "successful" if the
compressed version of the datum is smaller than the uncompressed
version. I think this is overly generous: if compression reduces the
size of the datum by, say, 0.01%, it is likely a net loss to use the
compressed version of the datum since we'll need to pay for LZ
decompression every time that we de-TOAST it. This situation can occur
frequently when storing "mostly-uncompressible" data (compressed images,
encrypted data, etc.) -- some parts of the data will compress well (e.g.
metadata), but the vast majority will not.

It's true that LZ decompression is fast, so we should probably use the
compressed version of the datum unless the reduction in size is very
small. I'm not sure precisely what that threshold should be, however.

Comments?

-Neil



Responses

pgsql-hackers by date

Next:From: Bruce MomjianDate: 2006-02-26 03:11:10
Subject: Re: TOAST compression
Previous:From: James William PyeDate: 2006-02-26 01:53:35
Subject: Re: Pl/Python -- current maintainer?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group