Different compression methods for FPI

From: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Different compression methods for FPI
Date: 2021-02-27 07:43:52
Message-ID: 3037310D-ECB7-4BF1-AF20-01C10BB33A33@yandex-team.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi!

There is a lot of different compression threads nearby. And that's great!
Every few bytes going to IO still deserve to be compressed.

Currently, we have a pglz compression for WAL full page images. As shown in [0] this leads to high CPU usage in pglz when wal_compression is on. Swapping pglz with lz4 increases pgbench tps by 21% on my laptop (if wal_compression is enabled).
So I think it worth to propose a patch to make wal_compression_method = {"pglz", "lz4", "zlib"}. Probably, "zstd" can be added to the list.

Even better option would be to teach WAL compression to compress everything, not only FPIs. But this is a big orthogonal chunk of work.

Attached is a draft taking CompressionId from "custom compression methods" patch and adding zlib to it.
I'm not sure where to add tests that check recovery with different methods. It seems to me that only TAP tests are suitable for this.

Thanks!

Best regards, Andrey Borodin.

[0] https://www.postgresql.org/message-id/323B1B01-DA42-419F-A99C-23E2C162D53B%40yandex-team.ru

Attachment Content-Type Size
0001-Use-different-compression-methods-for-FPIs.patch application/octet-stream 11.2 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2021-02-27 08:01:27 Re: Remove latch.c workaround for Linux < 2.6.27
Previous Message Peter Eisentraut 2021-02-27 07:16:13 Re: SEARCH and CYCLE clauses