Skip site navigation (1) Skip section navigation (2)

Re: TOAST compression

From: "Jim C(dot) Nasby" <jnasby(at)pervasive(dot)com>
To: Luke Lonergan <llonergan(at)greenplum(dot)com>
Cc: Neil Conway <neilc(at)samurai(dot)com>,PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: TOAST compression
Date: 2006-02-26 18:37:48
Message-ID: 20060226183748.GZ82012@pervasive.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Sun, Feb 26, 2006 at 09:31:05AM -0800, Luke Lonergan wrote:
> Note that this filesystem can do about 400MB/s, and we routinely see scan
> rates of 300MB/s within PG, so the real comparision is:
> 
> Direct seqscan at 300MB/s versus gunzip at 77.5MB/s

So the cutover point (on your system with very fast IO) is 4:1
compression (is that 20 or 25%?). But that's assuming that PostgreSQL
can read data as fast as dd, which we all know isn't the case. That's
also assuming a pretty top-notch IO subsystem. Based on that, I'd argue
that 10% is probably a better setting, though it would be good to test
an actual case (does dbt3 produce fields large enough to ensure that
most of them will be toasted?)

Given the variables involved, maybe it makes sense to add a GUC?
-- 
Jim C. Nasby, Sr. Engineering Consultant      jnasby(at)pervasive(dot)com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461

In response to

Responses

pgsql-hackers by date

Next:From: Luke LonerganDate: 2006-02-26 19:05:50
Subject: Re: TOAST compression
Previous:From: Andrew - SupernewsDate: 2006-02-26 18:37:46
Subject: Re: possible design bug with PQescapeString()

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group