Re: Zedstore - compressed in-core columnar storage

From: Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru>
To: Andreas Karlsson <andreas(at)proxel(dot)se>, Ashwin Agrawal <aagrawal(at)pivotal(dot)io>, Mark Kirkwood <mark(dot)kirkwood(at)catalyst(dot)net(dot)nz>
Cc: PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Zedstore - compressed in-core columnar storage
Date: 2019-04-11 13:52:33
Message-ID: 59ef6173-6667-8a02-ff7c-a52c8ce05a4f@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 11.04.2019 16:18, Andreas Karlsson wrote:
>
> On 4/11/19 10:46 AM, Konstantin Knizhnik wrote:
>
>> This my results of compressing pbench data using different compressors:
>>
>> Configuration Size (Gb) Time (sec)
>> no compression
>> 15.31 92
>> zlib (default level) 2.37 284
>> zlib (best speed) 2.43 191
>> postgres internal lz 3.89 214
>> lz4 4.12
>> 95
>> snappy 5.18 99
>> lzfse 2.80 1099
>> (apple) 2.80 1099
>> 1.69 125
>>
>>
>>
>> You see that zstd provides almost 2 times better compression ration
>> and almost at the same speed.
>
>
> What is "(apple) 2.80 1099"? Was that intended to be zstd?
>
> Andreas
>
Ugh...
Cut and paste problems.
The whole document can be found here:
http://garret.ru/PageLevelCompression.pdf

lzfse (apple)       2.80    1099
zstd (facebook)  1.69    125

ztsd is compression algorithm proposed by facebook:
https://github.com/facebook/zstd
Looks like it provides the best speed/compress ratio result.

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2019-04-11 13:56:37 Re: [PATCH v20] GSSAPI encryption support
Previous Message Alvaro Herrera 2019-04-11 13:49:47 Re: REINDEX CONCURRENTLY 2.0