Re: pglz compression performance, take two

> On Jan 28, 2021, at 2:56 AM, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>
>
>
>> 22 янв. 2021 г., в 07:48, Justin Pryzby <pryzby(at)telsasoft(dot)com> написал(а):
>>
>> @cfbot: rebased
>> <0001-Reorganize-pglz-compression-code.patch>
>
> Thanks!
>
> I'm experimenting with TPC-C over PostgreSQL 13 on production-like cluster in the cloud. Overall performance is IO-bound, but compression is burning a lot energy too (according to perf top). Cluster consists of 3 nodes(only HA, no standby queries) with 32 vCPU each, 128GB RAM, sync replication, 2000 warehouses, 240GB PGDATA.
>
> Samples: 1M of event 'cpu-clock', 4000 Hz, Event count (approx.): 177958545079
> Overhead Shared Object Symbol
> 18.36% postgres [.] pglz_compress
> 3.88% [kernel] [k] _raw_spin_unlock_irqrestore
> 3.39% postgres [.] hash_search_with_hash_value
> 3.00% [kernel] [k] finish_task_switch
> 2.03% [kernel] [k] copy_user_enhanced_fast_string
> 1.14% [kernel] [k] filemap_map_pages
> 1.02% postgres [.] AllocSetAlloc
> 0.93% postgres [.] _bt_compare
> 0.89% postgres [.] PinBuffer
> 0.82% postgres [.] SearchCatCache1
> 0.79% postgres [.] LWLockAttemptLock
> 0.78% postgres [.] GetSnapshotData
>
> Overall cluster runs 862tps (52KtpmC, though only 26KtmpC is qualified on 2K warehouses).
>
> Thanks!

Robert Haas just committed Dilip Kumar's LZ4 compression, bbe0a81db69bd10bd166907c3701492a29aca294.

Is this pglz compression patch still relevant? How does the LZ4 compression compare on your hardware?

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company