Re: Add LZ4 compression in pg_dump

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: gkokolatos(at)pm(dot)me
Cc: Michael Paquier <michael(at)paquier(dot)xyz>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Rachel Heaton <rachelmheaton(at)gmail(dot)com>
Subject: Re: Add LZ4 compression in pg_dump
Date: 2023-01-15 22:32:11
Message-ID: 20230115223211.GG9837@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sat, Jan 14, 2023 at 03:43:08PM -0600, Justin Pryzby wrote:
> On Sun, Jan 08, 2023 at 01:45:25PM -0600, Justin Pryzby wrote:
> > pg_compress_specification is being passed by value, but I think it
> > should be passed as a pointer, as is done everywhere else.
>
> ISTM that was an issue with 5e73a6048, affecting a few public and
> private functions. I wrote a pre-preparatory patch which changes to
> pass by reference.

I updated 001 to change SetOutput() to pass by reference, too (before,
that ended up in the 002 patch).

I can't see any issue in 002 other than the == GZIP change (the fix for
which I'd previously included in a later patch).

> One more thing: some comments still refer to the cfopen API, which this
> patch removes.
>
> > There were "LZ4" comments and file extension stuff in the preparatory
> > commit. But now it seems like you *removed* them in the LZ4 commit
> > (where it actually belongs) rather than *moving* it from the
> > prior/parent commit *to* the lz4 commit. I recommend to run something
> > like "git diff @{1}" whenever doing this kind of patch surgery.
>
> TODO

I addressed that in the fixup commits 005 and 007.

--
Justin

Attachment Content-Type Size
0001-pg_dump-pass-pg_compress_specification-as-a-pointer.patch text/x-diff 13.8 KB
0002-Prepare-pg_dump-internals-for-additional-compression.patch text/x-diff 22.6 KB
0003-f.patch text/x-diff 786 bytes
0004-Introduce-Compressor-API-in-pg_dump.patch text/x-diff 65.4 KB
0005-f.patch text/x-diff 5.0 KB
0006-Add-LZ4-compression-in-pg_-dump-restore.patch text/x-diff 28.8 KB
0007-f.patch text/x-diff 12.1 KB
0008-TMP-pg_dump-use-lz4-by-default-for-CI-only.patch text/x-diff 2.0 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tatsuo Ishii 2023-01-15 22:35:20 test
Previous Message Tom Lane 2023-01-15 22:18:26 Re: logrep stuck with 'ERROR: int2vector has too many elements'