Re: Adding pg_dump flag for parallel export to pipes

On Fri, May 22, 2026 at 4:04 PM solai v <solai(dot)cdac(at)gmail(dot)com> wrote:
> I then tested the patch introducing --pipe support. The feature is
> quite useful for modern workflows where users want to stream dump
> output directly to compression or upload pipelines without relying on
> intermediate storage.

Thank you for the feedback.

> gzip: dump.gz: unexpected end of file
> This suggests that concurrent writes to a shared output target are not
> coordinated and can result in invalid dumps. It would be helpful to
> clarify expected usage patterns here. For example: whether users are
> expected to generate distinct outputs per worker, or whether
> safeguards should be implemented to prevent multiple workers from
> writing to the same destination.

I added a warning for cases where the pipe command provided with
parallel dump and restore doesn't contain a `%f`. We can also add it
to the documentation. Let me know what you think.

> scenarios I observed backend logs such as:
> FATAL: connection to client lost
> Broken pipe
> While this is expected when the pipe terminates prematurely, it may be
> worth considering whether error messaging or cleanup behavior can be
> made clearer from the user perspective.

I added the failed command to the error message. I'm not sure if we
can do any auto-cleanup commands which succeeded.

Thanks & Regards

Nitin Motiani
Google