Re: pg_basebackup's --gzip switch misbehaves

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: pg_basebackup's --gzip switch misbehaves
Date: 2022-09-22 00:31:48
Message-ID: 20220922003148.GK31833@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Sep 13, 2022 at 04:13:20PM +0900, Michael Paquier wrote:
> diff --git a/src/common/compression.c b/src/common/compression.c
> index da3c291c0f..ac26287d54 100644
> --- a/src/common/compression.c
> +++ b/src/common/compression.c
> @@ -249,36 +299,49 @@ expect_integer_value(char *keyword, char *value, pg_compress_specification *resu
> char *
> validate_compress_specification(pg_compress_specification *spec)
> {
> + int min_level = 1;
> + int max_level = 1;
> + int default_level = 0;
> +
> /* If it didn't even parse OK, it's definitely no good. */
> if (spec->parse_error != NULL)
> return spec->parse_error;
>
> /*
> - * If a compression level was specified, check that the algorithm expects
> - * a compression level and that the level is within the legal range for
> - * the algorithm.
> + * Check that the algorithm expects a compression level and it is
> + * is within the legal range for the algorithm.
> */
> - if ((spec->options & PG_COMPRESSION_OPTION_LEVEL) != 0)
> + switch (spec->algorithm)
> {
> - int min_level = 1;
> - int max_level;
> -
> - if (spec->algorithm == PG_COMPRESSION_GZIP)
> + case PG_COMPRESSION_GZIP:
> max_level = 9;
> - else if (spec->algorithm == PG_COMPRESSION_LZ4)
> +#ifdef HAVE_LIBZ
> + default_level = Z_DEFAULT_COMPRESSION;
> +#endif
> + break;
> + case PG_COMPRESSION_LZ4:
> max_level = 12;
> - else if (spec->algorithm == PG_COMPRESSION_ZSTD)
> + default_level = 0; /* fast mode */
> + break;
> + case PG_COMPRESSION_ZSTD:
> max_level = 22;

I should've suggested to add:

> min_level = -7;

which has been supported since zstd 1.3.4 (and postgres requires 1.4.0).

I think at some point (maybe before releasing 1.3.4) the range was
increased to very large(small), negative levels. It's possible to query
the library about the lowest supported compression level, but then
there's a complication regarding the client-side library version vs the
server-side version. So it seems better to just use -7.

--
Justin

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David G. Johnston 2022-09-22 00:34:20 Re: pg_basebackup --create-slot-if-not-exists?
Previous Message Ashwin Agrawal 2022-09-22 00:07:06 pg_basebackup --create-slot-if-not-exists?