| From: | Nitin Motiani <nitinmotiani(at)google(dot)com> |
|---|---|
| To: | solai v <solai(dot)cdac(at)gmail(dot)com> |
| Cc: | Hannu Krosing <hannuk(at)google(dot)com>, Mahendra Singh Thalor <mahi6run(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: Adding pg_dump flag for parallel export to pipes |
| Date: | 2026-08-08 10:25:00 |
| Message-ID: | CAH5HC94uWqwGwWzCYHB_h4-jt1dT_M1=th+3mJaxnAdZ5XBAMg@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Fri, Jun 12, 2026 at 10:50 AM solai v <solai(dot)cdac(at)gmail(dot)com> wrote:
>
> without creating an intermediate dump directory.
> I verified error handling (--pipe='invalid_cmd') as well and correctly
> reported the error as "pg_dump: error: pipe command failed:
> "invalid_cmd": command not found".
> Similarly, --pipe='gzip | false' correctly propagated the child
> process failure. I also performed an end-to-end validation of the new
> functionality by restoring the generated archive using pg_restore
> --pipe. The restore completed successfully, and row counts for the
> test tables matched the original database confirming that dump
> generation, compression, restore, and data integrity all worked
> correctly. One particularly useful improvement I noticed is the
> handling of parallel jobs when the %f placeholder is omitted,
> producing the warning: "pg_dump: warning: parallel jobs with --pipe
> usually require the "%f" placeholder to avoid data corruption from
> multiple workers writing to the same file". This is an usability
> improvement, as it explicitly warns users against a common misuse that
> could otherwise lead to corrupted output when multiple workers write
> to the same destination file. Overall, the patch series worked well
> and the new --pipe support behaves correctly in my testing, also the
> newly added warning for missing %f significantly improves the user
> experience for parallel jobs. The patch looks good to me.
>
>
Thanks for the feedback. Rebasing and uploading v18.
| Attachment | Content-Type | Size |
|---|---|---|
| v18-0003-Fixes-and-refactors-in-pipe-command.patch | application/x-patch | 35.9 KB |
| v18-0002-Add-pipe-command-support-in-pg_restore.patch | application/x-patch | 6.0 KB |
| v18-0004-Add-tests-for-pipe.patch | application/x-patch | 22.1 KB |
| v18-0001-Add-pipe-command-support-for-directory-mode-of-p.patch | application/x-patch | 30.6 KB |
| v18-0005-Add-documentation-for-pipe-in-pg_dump-and-pg_res.patch | application/x-patch | 7.3 KB |
| From | Date | Subject | |
|---|---|---|---|
| Previous Message | Henri GASC | 2026-08-08 09:53:21 | [SQL/PGQ] Native executor for Graph query |