Re: zstd compression for pg_dump

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: Kirk Wolak <wolakk(at)gmail(dot)com>
Cc: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Jacob Champion <jchampion(at)timescale(dot)com>, pgsql-hackers(at)postgresql(dot)org, gkokolatos(at)pm(dot)me, Michael Paquier <michael(at)paquier(dot)xyz>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Geoghegan <pg(at)bowt(dot)ie>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Dipesh Pandit <dipesh(dot)pandit(at)gmail(dot)com>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Mark Dilger <mark(dot)dilger(at)enterprisedb(dot)com>
Subject: Re: zstd compression for pg_dump
Date: 2023-03-29 13:35:28
Message-ID: ZCQ+oBeZir1x/9eS@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Mar 28, 2023 at 02:03:49PM -0400, Kirk Wolak wrote:
> On Tue, Mar 28, 2023 at 12:23 PM Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
> > On 3/27/23 19:28, Justin Pryzby wrote:
> > > On Fri, Mar 17, 2023 at 03:43:31AM +0100, Tomas Vondra wrote:
> > >> On 3/16/23 05:50, Justin Pryzby wrote:
> > >>> On Fri, Mar 10, 2023 at 12:48:13PM -0800, Jacob Champion wrote:
> > >>>> On Wed, Mar 8, 2023 at 10:59 AM Jacob Champion <jchampion(at)timescale(dot)com> wrote:
> > >>>>> I did some smoke testing against zstd's GitHub release on Windows. To
> > ...
> > OK. I don't have access to a Windows machine so I can't test that. Is it
> > possible to disable the zstd threading, until we figure this out?
>
> Thomas since I appear to be one of the few windows users (I use both), can I help?
> I can test pg_dump... for you, easy to do. I do about 5-10 pg_dumps a day
> on windows while developing.

It'd be great if you'd exercise this and other changes to
pg_dump/restore. Tomas just pushed a bugfix, so be sure to "git pull"
before testing, or else you might rediscover the bug.

If you have a zstd library with thread support, you could test with
-Z zstd:workers=3. But I think threads aren't enabled in the common
libzstd packages. Jacob figured out how to compile libzstd easily using
"meson wraps" - but I don't know the details.

--
Justin

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2023-03-29 14:28:16 Re: logical decoding and replication of sequences, take 2
Previous Message Dmitry Koval 2023-03-29 13:32:36 Re: Add SPLIT PARTITION/MERGE PARTITIONS commands