Re: [PoC PATCH] Parallel dump to /dev/null

From: Michael Banck <michael(dot)banck(at)credativ(dot)de>
To: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: [PoC PATCH] Parallel dump to /dev/null
Date: 2018-02-02 09:44:01
Message-ID: 1517564641.30021.7.camel@credativ.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Am Freitag, den 02.02.2018, 13:22 +0900 schrieb Michael Paquier:
> On Thu, Feb 01, 2018 at 02:24:46PM +0100, Michael Banck wrote:
> > dumping a database to /dev/null via pg_dump is (AFAIK) one recommended
> > way to check for corruption. However, dumping to /dev/null is currently
> > not supported in directory mode which makes it not possible to dump to
> > /dev/null in parallel.
>
> FWIW, I use this trick as well in some test deployments. Now, those
> deployments happen in places where I want the checks to warn me, so the
> time it takes is not really an issue generally. Do folks here think
> that speedups would be worth it? Say to make the operation shorter on
> production systems for example. A lengthy pg_dump bloats data for a
> longer time, so I can buy that shorter times may help, though I think
> that hearing from other folks is necessary as well.

Another use-case would be automatic restores of basebackups, where you
might want to dump to /dev/null afterwards to check for issues, as just
starting the basebackup wouldn't tell you. If you have lots of instances
and lots of basebackups to check, doing that in parallel might be
helpful.

Michael

--
Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax: +49 2166 9901-100
Email: michael(dot)banck(at)credativ(dot)de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Pierre Ducroquet 2018-02-02 09:48:16 Re: JIT compiling with LLVM v9.1
Previous Message Amit Langote 2018-02-02 09:39:34 Re: Boolean partitions syntax