From:
"Robert Haas" <robertmhaas(at)gmail(dot)com>
To:
"Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>
Cc:
"Alex Hunsaker" <badalex(at)gmail(dot)com>, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject:
Re: Significantly larger toast tables on 8.4?
Date:
2009-01-02 20:35:18
Message-ID:
603c8f070901021235t1182a798g6526a46202d6a6cf@mail.gmail.com (view raw or flat )
Thread:
2009-01-02 04:30:59 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-02 04:49:55 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-02 05:44:49 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-02 06:09:35 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-02 14:57:46 from "Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>
2009-01-02 16:01:56 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2009-01-02 17:44:38 from "Robert Haas" <robertmhaas(at)gmail(dot)com>
2009-01-02 18:44:48 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2009-01-02 19:50:34 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-03 01:30:30 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2009-01-03 02:27:46 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-03 09:46:30 from Simon Riggs <simon(at)2ndQuadrant(dot)com>
2009-01-02 19:42:56 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-02 20:23:13 from "Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>
2009-01-02 20:35:18 from "Robert Haas" <robertmhaas(at)gmail(dot)com>
2009-01-02 21:19:00 from "Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>
2009-01-02 21:43:40 from "Robert Haas" <robertmhaas(at)gmail(dot)com>
2009-01-03 01:36:16 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2009-01-03 02:37:31 from "Robert Haas" <robertmhaas(at)gmail(dot)com>
2009-01-03 03:30:31 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2009-01-03 04:34:11 from "Robert Haas" <robertmhaas(at)gmail(dot)com>
2009-01-03 05:17:16 from Andrew Chernow <ac(at)esilo(dot)com>
2009-01-03 06:16:58 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-03 22:56:13 from Peter Eisentraut <peter_e(at)gmx(dot)net>
2009-01-02 22:48:15 from Martijn van Oosterhout <kleptog(at)svana(dot)org>
2009-01-07 14:44:51 from "Gregory Maxwell" <gmaxwell(at)gmail(dot)com>
2009-01-02 20:59:35 from Gregory Stark <stark(at)enterprisedb(dot)com>
2009-01-03 01:46:21 from Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
2009-01-03 06:32:00 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-05 15:57:21 from "Merlin Moncure" <mmoncure(at)gmail(dot)com>
2009-01-03 10:46:53 from "Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>
2009-01-04 03:47:23 from Philip Warner <pjw(at)rhyme(dot)com(dot)au>
2009-01-04 04:29:02 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-04 04:56:57 from Philip Warner <pjw(at)rhyme(dot)com(dot)au>
2009-01-04 05:15:09 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-03 22:45:44 from Peter Eisentraut <peter_e(at)gmx(dot)net>
2009-01-04 22:42:05 from James Mansion <james(at)mansionfamily(dot)plus(dot)com>
2009-01-05 01:34:07 from "Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>
2009-01-05 07:38:21 from Peter Eisentraut <peter_e(at)gmx(dot)net>
2009-01-05 16:45:49 from Alvaro Herrera <alvherre(at)commandprompt(dot)com>
2009-01-05 16:55:03 from "Merlin Moncure" <mmoncure(at)gmail(dot)com>
2009-01-06 18:10:31 from Peter Eisentraut <peter_e(at)gmx(dot)net>
2009-01-06 19:57:41 from "Stephen R(dot) van den Berg" <srb(at)cuci(dot)nl>
2009-01-06 20:30:11 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
2009-01-02 18:56:47 from "Alex Hunsaker" <badalex(at)gmail(dot)com>
Lists:
pgsql-hackers
On Fri, Jan 2, 2009 at 3:23 PM, Stephen R. van den Berg <srb(at)cuci(dot)nl> wrote:
> Alex Hunsaker wrote:
>>I think we could just add another toast storage type: alter table
>>alter column set storage compress; ? It seems overkill to expose
>>PGLZ_Strategy knobs per column...
>
> Three things:
> a. Shouldn't it in theory be possible to have a decompression algorithm
> which is IO-bound because it decompresses faster than the disk can
> supply the data? (On common current hardware).
> b. Has the current algorithm been carefully benchmarked and/or optimised
> and/or chosen to fit the IO-bound target as close as possible?
> c. Are there any well-known pitfalls/objections which would prevent me from
> changing the algorithm to something more efficient (read: IO-bound)?
Any compression algorithm is going to require you to decompress the
entire string before extracting a substring at a given offset. When
the data is uncompressed, you can jump directly to the offset you want
to read. Even if the compression algorithm requires no overhead at
all, it's going to make the location of the data nondeterministic, and
therefore force additional disk reads.
...Robert
In response to
Responses
pgsql-hackers by date
Next :From: Gregory StarkDate: 2009-01-02 20:40:49Subject : Re: posix_fadvise v22Previous :From : Stephen R. van den BergDate : 2009-01-02 20:23:13Subject : Re: Significantly larger toast tables on 8.4?
