Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: tgl(at)sss(dot)pgh(dot)pa(dot)us
Cc: fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp, thomas(dot)munro(at)enterprisedb(dot)com, eric(dot)cyr(at)gmail(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT
Date: 2018-11-19 03:08:11
Message-ID: 20181119.120811.18214843.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

At Sat, 17 Nov 2018 11:14:49 -0500, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote in <23489(dot)1542471289(at)sss(dot)pgh(dot)pa(dot)us>
> I wrote:
> > After still further thought, it seems like "if (bytesread == 0)"
> > is the right way to proceed. That protects us against incorrectly
> > setting reached_eof if the pipe is being run in unbuffered or
> > line-buffered mode. (Which copy.c doesn't do, at the moment,
> > but I'd just as soon this code didn't need to assume that.
> > Also, I'm not 100% convinced that Windows or other platforms
> > might not act that way.) In the case where we terminate early
> > because we saw an in-band EOF marker, we would also not set reached_eof,
> > and again that seems like a good outcome: if we see SIGPIPE it
> > must mean that the program kept sending data after the EOF marker,
> > and that seems like a good thing to complain about. (It's only
> > guaranteed to fail if the program sends more than the current pipe
> > buffer-ful, but I don't feel a need to add extra code to check for
> > nonempty buffer contents as we exit.)
>
> Oh, wait, that last bit is backwards: if we see an in-band EOF mark,
> and as a consequence exit without having set reached_eof, then the
> exit code will think that SIGPIPE is ignorable. So transmitting
> more data after an EOF mark will not be complained of, whether
> it's within the same bufferload or not.
>
> Still, I can't get very excited about that. Potentially we could
> improve it by having the places that recognize EOF marks set
> reached_eof, but I'm unconvinced that it's worth the trouble.
> I'm thinking that it's better to err in the direction of reporting
> SIGPIPE less often not more often, considering that up to now
> we've never reported it at all and there've been so few complaints.

My opinion here is when we execute an external program on the
other end of a pipe, we should behave as loosely (tolerantly) as
ordinary un*x programs are expected. If we're connecting to
another PostgreSQL server, we should be stringent as the current
behavior.

In other words, we don't need to change the behavior of other
than the COPY_FILE case, but ClosePipeToProgram shouldn't
complain not only for SIGPIPE but any kinds of error.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Etsuro Fujita 2018-11-19 04:05:43 Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT
Previous Message Tom Lane 2018-11-18 04:45:41 Re: BUG #15511: Drop table error "invalid argument"

Browse pgsql-hackers by date

  From Date Subject
Next Message Richard Guo 2018-11-19 03:21:40 Re: Pull up sublink of type 'NOT NOT (expr)'
Previous Message Amit Langote 2018-11-19 02:43:19 Re: ATTACH/DETACH PARTITION CONCURRENTLY