Skip site navigation (1) Skip section navigation (2)

Re: Are there plans to add data compression feature to postgresql?

From: Gregory Stark <stark(at)enterprisedb(dot)com>
To: "Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com>
Cc: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, 小波 顾 <guxiaobo1982(at)hotmail(dot)com>, Grzegorz Jaśkiewicz <gryzman(at)gmail(dot)com>, chris(dot)ellis(at)shropshire(dot)gov(dot)uk, "pgsql-general\(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Re: Are there plans to add data compression feature to postgresql?
Date: 2008-10-31 00:03:39
Message-ID: 87k5bpaaw4.fsf@oxford.xeocode.com (view raw or flat)
Thread:
Lists: pgsql-general
"Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com> writes:

> On Thu, Oct 30, 2008 at 4:41 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> "Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com> writes:
>>> On Thu, Oct 30, 2008 at 4:01 PM, Gregory Stark <stark(at)enterprisedb(dot)com> wrote:
>>>> I can't really see trusting Postgres on a filesystem that felt free to
>>>> compress portions of it. Would the filesystem still be able to guarantee that
>>>> torn pages won't "tear" across adjacent blocks? What about torn pages that
>>>> included hint bits being set?
>>
>>> I can't see PostgreSQL noticing it. PostgreSQL hands the OS a 512byte
>>> block, the OS compresses it and it's brethren as the go to disk,
>>> uncompresses as they come out, and as long as what you put in is what
>>> you get back it shouldn't really matter.
>>
>> I think Greg's issue is exactly about what guarantees you'll have left
>> after the data that comes back fails to be the data that went in.
>
> Sounds kinda hand wavy to me.  If compressed file systems didn't give
> you back what you gave them I couldn't imagine them being around for
> very long.

I don't know, NFS has lasted quite a while.

So you tell me, I write 512 bytes of data to a compressed filesystem, how does
it handle the torn page problem? Is it going to have to WAL log all data
operations again?

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com
  Ask me about EnterpriseDB's 24x7 Postgres support!

In response to

Responses

pgsql-general by date

Next:From: Scott MarloweDate: 2008-10-31 00:12:29
Subject: Re: Are there plans to add data compression feature to postgresql?
Previous:From: Noah FreireDate: 2008-10-30 23:59:28
Subject: Re: autovacuum

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group