Re: WAL compression setting after PostgreSQL LZ4 default change

From: Tomas Vondra <tomas(at)vondra(dot)me>
To: Christoph Berg <myon(at)debian(dot)org>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, wenhui qiu <qiuwenhuifx(at)gmail(dot)com>, Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>, Andres Freund <andres(at)anarazel(dot)de>, Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>, Michael Paquier <michael(at)paquier(dot)xyz>, Japin Li <japinli(at)hotmail(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, John Naylor <johncnaylorls(at)gmail(dot)com>
Subject: Re: WAL compression setting after PostgreSQL LZ4 default change
Date: 2026-07-31 21:44:30
Message-ID: 6d594ed0-5705-4026-a30a-2c36a07a970d@vondra.me
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 7/31/26 17:50, Christoph Berg wrote:
> Re: Andrey Borodin
>> FWIW on 19beta2, one client: a seqscan setting hint bits on a freshly
>> checkpointed 380MB table with wal_log_hints on, so ~47k full-page images
>> and little else. Best of three: off 765ms, pglz 1402ms, lz4 444ms, zstd
>> 797ms, writing 382/105/107/67 MB of WAL (fsync off, so the I/O saving is
>> understated). lz4 is the only setting faster than no compression at all,
>> while pglz costs nearly double for the size lz4 gives away for free. zstd
>> compresses a third better but pays for it in CPU, partly by allocating a
>> ZSTD_CCtx per FPI - see the nearby thread "Compression of bigger WAL
>> records". +1 for "on" meaning lz4 where available.
>
> v4 moves lz4 first.
>
> (I did not include any reason in the commit message about choosing lz4
> over zstd yet.)
>
> One minor bit: The patch now has
>
> #if defined(USE_LZ4)
> #define DEFAULT_WAL_COMPRESSION·WAL_COMPRESSION_LZ4
> #elif defined(USE_ZSTD)
> #define DEFAULT_WAL_COMPRESSION·WAL_COMPRESSION_ZSTD
>
> when the old one had
> #ifdef USE_ZSTD
> #elif defined(USE_LZ4)
>
> I left it at "#if defined(USE_LZ4)" because that looks more
> symmetrical.
>

Thanks. FWIW I've brought this up with the RMT, and there seem to be no
objections to getting this done for PG19. Unless someone objects in the
next couple days, I'll get it committed.

As for the zstd vs. lz4 - the patch put zstd first mostly because
Michael prefers it over lz4. I don't think either of these is superior.
AFAIK the algorithms make different trade offs - zstd tends to compress
better, but uses (substantially) more CPU. I think it makes sense to
default to lz4, it's clearly better than pglz, with low CPU usage.

regards

--
Tomas Vondra

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Melanie Plageman 2026-07-31 21:53:13 Re: pg19b1: TRAP: failed Assert("pgstat_bktype_io_stats_valid(bktype_shstats, MyBackendType)")
Previous Message Melanie Plageman 2026-07-31 21:33:32 Re: pg19b1: TRAP: failed Assert("pgstat_bktype_io_stats_valid(bktype_shstats, MyBackendType)")