Skip site navigation (1) Skip section navigation (2)

Re: Improve compression speeds in pg_lzcompress.c

From: Takeshi Yamamuro <yamamuro(dot)takeshi(at)lab(dot)ntt(dot)co(dot)jp>
To: Greg Stark <stark(at)mit(dot)edu>
Cc: John R Pierce <pierce(at)hogranch(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Improve compression speeds in pg_lzcompress.c
Date: 2013-01-08 09:19:13
Message-ID: 50EBE491.5050700@lab.ntt.co.jp (view raw or flat)
Thread:
Lists: pgsql-hackers
Hi,

(2013/01/07 22:36), Greg Stark wrote:
> On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce<pierce(at)hogranch(dot)com>  wrote:
>> On 1/7/2013 2:05 AM, Andres Freund wrote:
>>>
>>> I think there should be enough bits available in the toast pointer to
>>> indicate the type of compression. I seem to remember somebody even
>>> posting a patch to that effect?
>>> I agree that it's probably too late in the 9.3 cycle to start with this.
>>
>>
>> so an upgraded database would have old toasted values in the old compression
>> format, and new toasted values in the new format in an existing table?
>> that's kind of ugly.
>
> I haven't looked at the patch. It's not obvious to me from the
> description that the output isn't backwards compatible. The way the LZ
> toast compression works the output is self-describing. There are many
> different outputs that would decompress to the same thing and the
> compressing code can choose how hard to look for earlier matches and
> when to just copy bytes wholesale but the decompression will work
> regardless.

My patch is not backwards compatible, so we need some features
to switch these old and new disk formats.

I think the discussion below is helpful in this use.
That is, PGLZ_Header is used as this purpose.
http://archives.postgresql.org/pgsql-hackers/2012-03/msg00971.php

regards,
-- 
----
Takeshi Yamamuro
NTT Cyber Communications Laboratory Group
Software Innovation Center
(Open Source Software Center)
Tel: +81-3-5860-5057 Fax: +81-3-5463-5490
Mail:yamamuro(dot)takeshi(at)lab(dot)ntt(dot)co(dot)jp


In response to

Responses

pgsql-hackers by date

Next:From: Magnus HaganderDate: 2013-01-08 09:32:40
Subject: Re: PL/Python result object str handler
Previous:From: Pavan DeolaseeDate: 2013-01-08 09:11:54
Subject: Re: lazy_vacuum_heap()'s removal of HEAPTUPLE_DEAD tuples

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group