lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)

From: Christoph Berg <myon(at)debian(dot)org>
To: Tomas Vondra <tomas(dot)vondra(at)postgresql(dot)org>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
Date: 2023-03-08 17:55:03
Message-ID: ZAjL96N9ZW84U59p@msg.df7cb.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

Re: Tomas Vondra
> Add LZ4 compression to pg_dump

This broke the TAP tests on Ubuntu 18.04 (bionic):

[17:06:45.513](0.000s) ok 1927 - compression_lz4_custom: should not dump test_table with 4-row INSERTs
# Running: pg_dump --jobs=2 --format=directory --compress=lz4:1 --file=/home/myon/projects/postgresql/pg/master/build/src/bin/pg_dump/tmp_check/tmp_test__aAO/compression_lz4_dir postgres
[17:06:46.651](1.137s) ok 1928 - compression_lz4_dir: pg_dump runs
# Running: /usr/bin/lz4 -z -f --rm /home/myon/projects/postgresql/pg/master/build/src/bin/pg_dump/tmp_check/tmp_test__aAO/compression_lz4_dir/blobs.toc /home/myon/projects/postgresql/pg/master/build/src/bin/pg_dump/tmp_check/tmp_test__aAO/compression_lz4_dir/blobs.toc.lz4
Incorrect parameters
Usage :
/usr/bin/lz4 [arg] [input] [output]

input : a filename
with no FILE, or when FILE is - or stdin, read standard input
Arguments :
-1 : Fast compression (default)
-9 : High compression
-d : decompression (default for .lz4 extension)
-z : force compression
-f : overwrite output without prompting
-h/-H : display help/long help and exit
[17:06:46.667](0.016s) not ok 1929 - compression_lz4_dir: compression commands
[17:06:46.668](0.001s)
[17:06:46.668](0.001s) # Failed test 'compression_lz4_dir: compression commands'
[17:06:46.669](0.000s) # at t/002_pg_dump.pl line 4274.
[17:06:46.670](0.001s) ok 1930 - compression_lz4_dir: glob check for /home/myon/projects/postgresql/pg/master/build/src/bin/pg_dump/tmp_check/tmp_test__aAO/compression_lz4_dir/toc.dat

The lz4 binary there doesn't have the --rm option yet.

liblz4-tool 0.0~r131-2ubuntu3

--rm appears in a single place only:

# Give coverage for manually compressed blob.toc files during
# restore.
compress_cmd => {
program => $ENV{'LZ4'},
args => [
'-z', '-f', '--rm',
"$tempdir/compression_lz4_dir/blobs.toc",
"$tempdir/compression_lz4_dir/blobs.toc.lz4",
],
},

18.04 will be EOL in a few weeks so it might be ok to just say it's
not supported, but removing the input file manually after calling lz4
would be an easy fix.

Christoph

In response to

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Andres Freund 2023-03-08 19:20:13 pgsql: meson: Add target for installing test files & improve install_te
Previous Message Alvaro Herrera 2023-03-08 17:42:35 pgsql: 001_libpq_pipeline.pl: use Test::Differences if available

Browse pgsql-hackers by date

  From Date Subject
Next Message Antonin Houska 2023-03-08 18:07:57 Re: Parallelize correlated subqueries that execute within each worker
Previous Message Jeff Davis 2023-03-08 17:45:55 Re: allow meson to find ICU in non-standard localtion