Re: Compression

On Thu, Apr 14, 2011 at 7:42 PM, Adrian Klaver <adrian(dot)klaver(at)gmail(dot)com> wrote:
> On Thursday, April 14, 2011 5:51:21 pm Yang Zhang wrote:
>
>> >
>
>> > adrian(dot)klaver(at)gmail(dot)com
>
>>
>
>> Already know about TOAST. I could've been clearer, but that's not the
>
>> same as the block-/page-level compression I was referring to.
>
> I am obviously missing something. The TOAST mechanism is designed to keep
> tuple data below the default 8KB page size. In fact it kicks in at a lower
> level than that:
>
> "The TOAST code is triggered only when a row value to be stored in a table
> is wider than TOAST_TUPLE_THRESHOLD bytes (normally 2 kB). The TOAST code
> will compress and/or move field values out-of-line until the row value is
> shorter than TOAST_TUPLE_TARGET bytes (also normally 2 kB) or no more gains
> can be had. During an UPDATE operation, values of unchanged fields are
> normally preserved as-is; so an UPDATE of a row with out-of-line values
> incurs no TOAST costs if none of the out-of-line values change.'
>
> Granted no all data types are TOASTable. Are you looking for something more
> aggressive than that?

Yes.

http://blog.oskarsson.nu/2009/03/hadoop-feat-lzo-save-disk-space-and.html

http://wiki.apache.org/hadoop/UsingLzoCompression

http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-compression-internals-algorithms.html

>
> --
>
> Adrian Klaver
>
> adrian(dot)klaver(at)gmail(dot)com

--
Yang Zhang
http://yz.mit.edu/