Re: PGLZ Compression Optimization

On 7/16/26 14:16, Daniel Gustafsson wrote:
>> On 13 Jul 2026, at 20:17, Tomas Vondra <tomas(at)vondra(dot)me> wrote:
>> On 7/13/26 13:15, wenhui qiu wrote:
>
>>> The patch also optimizes the legacy path with a rolling hash, indexed
>>> history rings, bounded candidate searches, word/SIMD match extension,
>>> and an AVX2 implementation selected by PostgreSQL's runtime CPU check.
>>> Other platforms use PostgreSQL's existing SIMD abstraction.
>>>
>>
>> This, on the other hand would be a *huge* improvement for all existing
>> systems with a lot of data compressed with pglz (and there's fair amount
>> of those). If we could just make them faster by optimizing the code,
>> that seems very desirable.
>
> +1, if optimizing the code can unlock performance improvements while being
> backwards compatible it would be a very big win.
>

100% agreed. Introducing new compression schemes into PGLZ is one thing
(and I think it'd be wasted effort). But making the current scheme
faster (particularly the decompression part) would be a huge benefit for
existing systems that can't simply recompress everything.

I wonder if we could introduce a mechanism to automatically recompress
stuff in the background (with a background worker, similar to what
online checksums do). That'd allow systems to migrate to a new
compression algorithm. But that's a completely separate feature.

regards

--
Tomas Vondra