Re: Add LZ4 compression in pg_dump

From: gkokolatos(at)pm(dot)me
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Rachel Heaton <rachelmheaton(at)gmail(dot)com>
Subject: Re: Add LZ4 compression in pg_dump
Date: 2022-12-02 16:15:10
Message-ID: T33dciWfZ3WaU10Mspm2_vsftrl0DI_cs4rQcrT_pLubaLq_HCRlW5QhEOT1cWtSRtdeTVXrL2r2BTYz3Fjz_3YNh6i77lwd5AxNxb5oanQ=@pm.me
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


------- Original Message -------
On Friday, December 2nd, 2022 at 2:56 AM, Michael Paquier <michael(at)paquier(dot)xyz> wrote:

> On top of my mind, I briefly recall that -r is not that portable. And
> the toc format makes the files generated non-deterministic as these
> use OIDs..
>
> [.. thinks ..]
>
> We are going to need a new thing here, as compress_cmd cannot be
> directly used. What if we used only an array of glob()-able elements?
> Let's say "expected_contents" that could include a "dir_path/*.gz"
> conditional on $supports_gzip? glob() can only be calculated when the
> test is run as the file names cannot be known beforehand :/

You are very correct. However one can glob after the fact. Please find
0001 of the attached v14 which attempts to implement it.

> I have gone through the patch again, and applied it. Thanks!

Thank you. Please find the rest of of the patchset series rebased on top
of it. I dare to say that 0002 is in a state worth of your consideration.

Cheers,
//Georgios

> --
> Michael

Attachment Content-Type Size
v14-0001-Provide-coverage-for-pg_dump-default-compression.patch text/x-patch 1.7 KB
v14-0002-Prepare-pg_dump-internals-for-additional-compres.patch text/x-patch 19.9 KB
v14-0003-Introduce-Compressor-API-in-pg_dump.patch text/x-patch 56.4 KB
v14-0004-Add-LZ4-compression-in-pg_-dump-restore.patch text/x-patch 28.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2022-12-02 16:41:31 Re: [PoC] Improve dead tuple storage for lazy vacuum
Previous Message Reid Thompson 2022-12-02 16:09:30 Re: Add tracking of backend memory allocated to pg_stat_activity