| From: | Nitin Motiani <nitinmotiani(at)google(dot)com> |
|---|---|
| To: | Hannu Krosing <hannuk(at)google(dot)com> |
| Cc: | Mahendra Singh Thalor <mahi6run(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: Adding pg_dump flag for parallel export to pipes |
| Date: | 2026-05-04 04:39:13 |
| Message-ID: | CAH5HC97skKgTkOBUipZjpOCXYc27txdzFL8UmVmBKaRvc4LO5A@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi Mahendra,
I'm attaching the latest version of the patch. This incorporates
several suggestions Mahendra made. I've also fixed the failing test
and added support for Windows in the test by using perlbin instead of
cat. Here is the list of changes in this version:
1. Changed pipe-command to pip.
2. Added better error handling for invalid command.
3. Added shell escaping in the command before setting it as the file path.
4. I decided to change the mode to PG_BINARY_W for the large object
toc files even in the standard case. If there is a concern, we can
revert to the older version.
5. Added a bunch of new tests for various scenarios and parallelism.
Currently these changes are in a separate commit. After the review, I
can squash the first 3 commits of pg_dump, pg_restore, and the fixes
in one commit.
On Sat, Mar 14, 2026 at 10:37 PM Mahendra Singh Thalor
<mahi6run(at)gmail(dot)com> wrote:
>
> Comment5: I think we can support this new pipe option with pg_dumpall also as we support directory mode in pg_dumpall from v19.
Regarding this, I haven't looked into the details of how that would
work. If the rest look good, we can consider implementing this in
another patch. For the time being, I've skipped this option in
pg_dumpall and the global restore. Let me know what you think.
Thanks
Nitin Motiani,
Google
| Attachment | Content-Type | Size |
|---|---|---|
| v11-0003-Fixes-and-refactors-in-pipe-command.patch | application/x-patch | 40.2 KB |
| v11-0004-Add-tests-for-pipe.patch | application/x-patch | 14.9 KB |
| v11-0005-Add-documentation-for-pipe-in-pg_dump-and-pg_res.patch | application/x-patch | 7.4 KB |
| v11-0002-Add-pipe-command-support-in-pg_restore.patch | application/x-patch | 9.5 KB |
| v11-0001-Add-pipe-command-support-for-directory-mode-of-p.patch | application/x-patch | 31.8 KB |
| From | Date | Subject | |
|---|---|---|---|
| Previous Message | Tom Lane | 2026-05-04 04:38:38 | Re: Make printtup a bit faster |