Skip site navigation (1) Skip section navigation (2)

Re: Significantly larger toast tables on 8.4?

From: "Robert Haas" <robertmhaas(at)gmail(dot)com>
To: "Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>
Cc: "Alex Hunsaker" <badalex(at)gmail(dot)com>, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Significantly larger toast tables on 8.4?
Date: 2009-01-02 20:35:18
Message-ID: 603c8f070901021235t1182a798g6526a46202d6a6cf@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Fri, Jan 2, 2009 at 3:23 PM, Stephen R. van den Berg <srb(at)cuci(dot)nl> wrote:
> Alex Hunsaker wrote:
>>I think we could just add another toast storage type: alter table
>>alter column set storage compress; ?  It seems overkill to expose
>>PGLZ_Strategy knobs per column...
>
> Three things:
> a. Shouldn't it in theory be possible to have a decompression algorithm
>   which is IO-bound because it decompresses faster than the disk can
>   supply the data?  (On common current hardware).
> b. Has the current algorithm been carefully benchmarked and/or optimised
>   and/or chosen to fit the IO-bound target as close as possible?
> c. Are there any well-known pitfalls/objections which would prevent me from
>   changing the algorithm to something more efficient (read: IO-bound)?

Any compression algorithm is going to require you to decompress the
entire string before extracting a substring at a given offset.  When
the data is uncompressed, you can jump directly to the offset you want
to read.  Even if the compression algorithm requires no overhead at
all, it's going to make the location of the data nondeterministic, and
therefore force additional disk reads.

...Robert

In response to

Responses

pgsql-hackers by date

Next:From: Gregory StarkDate: 2009-01-02 20:40:49
Subject: Re: posix_fadvise v22
Previous:From: Stephen R. van den BergDate: 2009-01-02 20:23:13
Subject: Re: Significantly larger toast tables on 8.4?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group