Re: zstd compression for pg_dump

On 4/4/23 05:04, Justin Pryzby wrote:
> On Mon, Apr 03, 2023 at 11:26:09PM +0200, Tomas Vondra wrote:
>> On 4/3/23 21:17, Justin Pryzby wrote:
>>> On Sat, Apr 01, 2023 at 10:26:01PM +0200, Tomas Vondra wrote:
>>>>> Feel free to mess around with threads (but I'd much rather see the patch
>>>>> progress for zstd:long).
>>>>
>>>> OK, understood. The long mode patch is pretty simple. IIUC it does not
>>>> change the format, i.e. in the worst case we could leave it for PG17
>>>> too. Correct?
>>>
>>> Right, libzstd only has one "format", which is the same as what's used
>>> by the commandline tool. zstd:long doesn't change the format of the
>>> output: the library just uses a larger memory buffer to allow better
>>> compression. There's no format change for zstd:workers, either.
>>
>> OK. I plan to do a bit more review/testing on this, and get it committed
>> over the next day or two, likely including the long mode. One thing I
>> noticed today is that maybe long_distance should be a bool, not int.
>> Yes, ZSTD_c_enableLongDistanceMatching() accepts int, but it'd be
>> cleaner to cast the value during a call and keep it bool otherwise.
>
> Thanks for noticing. Evidently I wrote it using "int" to get the
> feature working, and then later wrote the bool parsing bits but never
> changed the data structure.
>
> This also updates a few comments, indentation, removes a useless
> assertion, and updates the warning about zstd:workers.
>

Thanks. I've cleaned up the 0001 a little bit (a couple comment
improvements), updated the commit message and pushed it. I plan to take
care of the 0002 (long distance mode) tomorrow, and that'll be it for
PG16 I think.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company