Re: WAL compression setting after PostgreSQL LZ4 default change

From: Tomas Vondra <tomas(at)vondra(dot)me>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: wenhui qiu <qiuwenhuifx(at)gmail(dot)com>, Christoph Berg <myon(at)debian(dot)org>, Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>, Michael Paquier <michael(at)paquier(dot)xyz>, Japin Li <japinli(at)hotmail(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, John Naylor <johncnaylorls(at)gmail(dot)com>
Subject: Re: WAL compression setting after PostgreSQL LZ4 default change
Date: 2026-07-31 12:55:56
Message-ID: 9ae034d4-294b-4d81-b2b3-c228bb2c1965@vondra.me
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 7/16/26 17:08, Andres Freund wrote:
> Hi,
>
> On 2026-07-16 17:00:58 +0200, Tomas Vondra wrote:
>> On 7/8/26 08:37, wenhui qiu wrote:
>>> Hi 
>>>
>>> > I would like to see "on" to be moved to some better value (likely lz4,
>>> > like toast compression), and if some day, there is yet a better
>>> > choice, we should move it again.
>>> Yes, After PostgreSQL 19, |lz4| became a required dependency unless
>>> you compile it yourself with |--without-lz4|. In most
>>> environments, |lz4| is already installed, so along this code path
>>> the effective choice is essentially fixed to |lz4|.At cluster scale,
>>> it would make more sense to expose this parameter in the frontend as
>>> a boolean setting, rather than letting users choose from multiple
>>> compression options.
>>>
>>>
>>> Thanks 
>>
>> I've been reminded of this proposal by a message in another thread [1],
>> which suggests this proposal (to adjust the wal_compression default) was
>> not accepted.
>>
>> Which seems strange to me, and I suspect it's a misunderstanding. From
>> the discussion in this thread it seems to me that:
>>
>> * the proposal is to change what 'on' means - currently it's always
>> pglz, but the proposal would change this to the "best" available
>> algorithm (so lz4)
>>
>> * the wal_compression default would remain "off"
>>
>> I think this proposal makes sense, and at least Michael and Christoph
>> seem to agree with that.
>
> +1 from me too.
>
>
> I've seen way too many incidents where wal_compression was very useful but due
> to the use of pglz slowed down crucial, single threaded, parts of the workload
> down so much that it had to be disabled again. We should not make it our
> users responsibility to know that the 'on' maps to an almost unusably slow
> compression algorithm.
>
>
> The compatibility concerns are *much* higher for toast compression than for
> wal compression, given that toast compression means that you have to build
> with the chosen compression pretty much forever, given that the compressed
> datum survive pg_upgrade. Whereas wal compression doesn't survive pg_upgrade (
> and you probably could even get away with just running with
> wal_compression=off for a while and then removing support for the relevant
> compression method).
>

Ack. What do you think about doing this for PG19? Or do you think that
ship has sailed at this point?

regards

--
Tomas Vondra

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2026-07-31 13:04:03 Re: WAL compression setting after PostgreSQL LZ4 default change
Previous Message Tomas Vondra 2026-07-31 12:52:07 Re: hashjoins vs. Bloom filters (yet again)