Re: Improve compression speeds in pg_lzcompress.c

Hi,

(2013/01/07 22:36), Greg Stark wrote:
> On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce<pierce(at)hogranch(dot)com> wrote:
>> On 1/7/2013 2:05 AM, Andres Freund wrote:
>>>
>>> I think there should be enough bits available in the toast pointer to
>>> indicate the type of compression. I seem to remember somebody even
>>> posting a patch to that effect?
>>> I agree that it's probably too late in the 9.3 cycle to start with this.
>>
>>
>> so an upgraded database would have old toasted values in the old compression
>> format, and new toasted values in the new format in an existing table?
>> that's kind of ugly.
>
> I haven't looked at the patch. It's not obvious to me from the
> description that the output isn't backwards compatible. The way the LZ
> toast compression works the output is self-describing. There are many
> different outputs that would decompress to the same thing and the
> compressing code can choose how hard to look for earlier matches and
> when to just copy bytes wholesale but the decompression will work
> regardless.

My patch is not backwards compatible, so we need some features
to switch these old and new disk formats.

I think the discussion below is helpful in this use.
That is, PGLZ_Header is used as this purpose.
http://archives.postgresql.org/pgsql-hackers/2012-03/msg00971.php

regards,
--
----
Takeshi Yamamuro
NTT Cyber Communications Laboratory Group
Software Innovation Center
(Open Source Software Center)
Tel: +81-3-5860-5057 Fax: +81-3-5463-5490
Mail:yamamuro(dot)takeshi(at)lab(dot)ntt(dot)co(dot)jp