Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>
Cc: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, thomas(dot)munro(at)enterprisedb(dot)com, eric(dot)cyr(at)gmail(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT
Date: 2018-11-16 17:33:51
Message-ID: 31355.1542389631@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

I wrote:
> Are we sufficiently convinced that we must have the dont-allow-partial
> option to not fix this in the back branches? I'm not.

I just had a thought about that: suppose we add a flag to CopyState
to indicate whether we reached EOF while reading. This wouldn't be
hugely expensive, just something like

switch (cstate->copy_dest)
{
case COPY_FILE:
bytesread = fread(databuf, 1, maxread, cstate->copy_file);
if (ferror(cstate->copy_file))
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not read from COPY file: %m")));
+ cstate->reached_eof |= (bytesread < maxread);
break;
case COPY_OLD_FE:

Then the logic in ClosePipeToProgram could be "if we did not reach
EOF, then a SIGPIPE termination is unsurprising. If we did reach
EOF, then SIGPIPE is an error." If the called program gets SIGPIPE
for some reason other than us closing the pipe early, then we would
see EOF next time we try to read, and correctly report that there's
a problem.

There are race-ish conditions in cases like the called program
getting an unrelated SIGPIPE at about the same time that we decide
to stop reading early, but I don't think it's really possible
to disambiguate that.

regards, tom lane

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Melanie Plageman 2018-11-16 18:31:47 Re: BUG #15160: planner overestimates number of rows in join when there are more than 200 rows coming from CTE
Previous Message Tom Lane 2018-11-16 17:20:51 Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

Browse pgsql-hackers by date

  From Date Subject
Next Message Laurenz Albe 2018-11-16 17:56:11 Re: Libpq support to connect to standby server as priority
Previous Message Tom Lane 2018-11-16 17:20:51 Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT