| From: | Michael Banck <michael(dot)banck(at)credativ(dot)de> |
|---|---|
| To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Subject: | [PoC PATCH] Parallel dump to /dev/null |
| Date: | 2018-02-01 13:24:46 |
| Message-ID: | 20180201132446.GA13784@nighthawk.caipicrew.dd-dns.de |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
dumping a database to /dev/null via pg_dump is (AFAIK) one recommended
way to check for corruption. However, dumping to /dev/null is currently
not supported in directory mode which makes it not possible to dump to
/dev/null in parallel.
I had a look at this, and it appears it would suffice to just override
the few spots in pg_backup_directory.c which assemble filenames in the
target directory to revert to '/dev/null' (or 'NUL' on Windows).
The attached proof-of-concept patch does that, and it seems to work; I'm
getting some nice speedups on a 170 GB test database:
$ time pg_dump -Fc -Z0 -f /dev/null TESTDB
real 32m45.120s
[...]
$ time pg_dump -Fd -j8 -Z0 -f /dev/null TESTDB
real 9m28.357s
Thoughts?
Michael
--
Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax: +49 2166 9901-100
Email: michael(dot)banck(at)credativ(dot)de
credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer
| Attachment | Content-Type | Size |
|---|---|---|
| parallel-pgdump-dev-null.patch | text/x-diff | 3.3 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Bruce Momjian | 2018-02-01 13:32:59 | Re: proposal: alternative psql commands quit and exit |
| Previous Message | Simon Riggs | 2018-02-01 13:16:05 | Re: MCV lists for highly skewed distributions |