pgsql: Support long distance matching for zstd compression

From: Tomas Vondra <tomas(dot)vondra(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Support long distance matching for zstd compression
Date: 2023-04-06 15:28:07
Message-ID: E1pkRWw-001h4Y-IA@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Support long distance matching for zstd compression

zstd compression supports a special mode for finding matched in distant
past, which may result in better compression ratio, at the expense of
using more memory (the window size is 128MB).

To enable this optional mode, use the "long" keyword when specifying the
compression method (--compress=zstd:long).

Author: Justin Pryzby
Reviewed-by: Tomas Vondra, Jacob Champion
Discussion: https://postgr.es/m/20230224191840.GD1653@telsasoft.com
Discussion: https://postgr.es/m/20220327205020.GM28503@telsasoft.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/2820adf7755d2a377546d5b55f5b1a4a39889336

Modified Files
--------------
doc/src/sgml/protocol.sgml | 10 ++++-
doc/src/sgml/ref/pg_basebackup.sgml | 4 +-
doc/src/sgml/ref/pg_dump.sgml | 2 +
src/backend/backup/basebackup_zstd.c | 12 ++++++
src/bin/pg_basebackup/bbstreamer_zstd.c | 13 ++++++
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 9 ++++-
src/bin/pg_dump/compress_zstd.c | 5 +++
src/bin/pg_dump/t/002_pg_dump.pl | 3 +-
src/bin/pg_verifybackup/t/008_untar.pl | 8 ++++
src/bin/pg_verifybackup/t/010_client_untar.pl | 8 ++++
src/common/compression.c | 57 ++++++++++++++++++++++++++-
src/include/common/compression.h | 2 +
12 files changed, 127 insertions(+), 6 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Justin Pryzby 2023-04-06 16:39:47 Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)
Previous Message Etsuro Fujita 2023-04-06 08:37:09 pgsql: postgres_fdw: Add support for parallel abort.