Re: Decompression bug in astreamer_lz4

From: Mikhail Gribkov <youzhick(at)gmail(dot)com>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Pg Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Decompression bug in astreamer_lz4
Date: 2025-06-27 12:28:18
Message-ID: CAMEv5_u0BWA+tcczoKYdBXMtkwHYfgZLoe0GsH_8pqNM6bc2iQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Michael,

> I think that it would be nice to have some test coverage for such
> cases in pg_verifybackup for both the client side and the server side
> of backups compressed, relying on the backend to generate some random
> data dumped into a file at the root of a data folder.

Yes, good point. Added checks to both client and server untar tests.
I also noticed that the problem showup depends on compression level. For
example in 008_untar.pl and 010_client_untar.pl tests it only showed up
with compression levels 0 and 1.
It looks to me that we do not have any chances to check all possible
combinations of vital parameters, but at least we'll check several cases
that we know make a difference.

> I am detecting a threshold close to 512kB to be able to reproduce the
> problem, so we don't need a file that large to see the failure, making
> such tests faster with less data required

I have raised the file size up to 640kB to make it a bit more reliable as
far as we are not sure about all lz4-internals that trigger the problem. It
still does not make the test data unreasonably large.

Attached is the updated version of the patch with test additions.

--
best regards,
Mikhail A. Gribkov

>

Attachment Content-Type Size
v2-Fix-lz4-decompressor.patch application/octet-stream 4.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Alexander Korotkov 2025-06-27 12:55:37 Re: Removing unneeded self joins
Previous Message Ryo Kanbayashi 2025-06-27 12:25:47 Re: [PATCH] PGSERVICEFILE as part of a normal connection string