Re: BUG #15554: Broken pipe when doing a COPY of a parallel query

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: luis(dot)carril(at)swarm64(dot)com
Cc: pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #15554: Broken pipe when doing a COPY of a parallel query
Date: 2018-12-17 14:50:16
Message-ID: 28346.1545058216@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

=?utf-8?q?PG_Bug_reporting_form?= <noreply(at)postgresql(dot)org> writes:
> when performing a COPY TO operation with the result of a parallel query a
> 'Broken Pipe' error is returned
> ...
> set force_parallel_mode to on;
> copy ( select 1) to program 'ls'; -- this SELECT also has a gather node
> ERROR: could not close pipe to external command: Broken pipe

I'm not convinced that this is a bug. The external program did not
read the data we sent it, and it seems appropriate to report that.

> Note that there is no broken pipe if there is no Gather node:
> set force_parallel_mode to on;
> copy ( select 1) to program 'ls';
> COPY 1

(I think you meant "off" not "on" there.) This is just a question of
timing: the parallel copy takes long enough to start up that the "ls"
has already done its thing and exited, so that we see EPIPE, while
the non-parallel case doesn't see the error. Ideally we'd report an
error in both cases, but the kernel doesn't make it possible to do
that :-(.

With a larger chunk of COPY data, the error is reported a bit sooner,
with different text:

regression=# copy ( select u from a) to program 'ls';
ERROR: could not write to COPY program: Broken pipe
regression=# copy ( select sum(u) from a) to program 'ls';
ERROR: could not close pipe to external command: Broken pipe

Perhaps it's worth adding an fflush at the end, and reporting any
error from the fflush using "could not write to COPY program".
But that seems cosmetic.

regards, tom lane

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Peter Geoghegan 2018-12-17 19:08:43 Re: BUG #15556: Duplicate key violations even when using ON CONFLICT DO UPDATE
Previous Message Luis Carril 2018-12-17 13:12:11 Re: BUG #15552: Unexpected error in COPY to a foreign table in a transaction