Re: [HACKERS] Trouble with COPY IN

On Jul 28, 2010, at 9:53 AM, Kris Jurka wrote:
> Technically you won't get NotificationResponse until transaction end, so you don't need to worry about that mid copy.

Ah, thanks for noting that. It would appear my original reading of the async section didn't get far enough beyond "Frontends must be prepared to deal with these messages at any time, even when not engaged in a query.". I see the note below clarifying NotificationResponse.

> One of the key points of confusion is that CopyData(EOF) does not result in an error.
> It results in ignoring any futher data.
> The problem I have is that for text mode it waits for CopyDone, but in binary mode it ends the copy sequence immediately.

That is bothersome. :\

> Additionally the interface exposed by the JDBC driver lets the user write arbitrary CopyData bytes to the server, so without parsing all of that we don't know whether they've issued CopyData(EOF) or not.

Okay, so you can't know with absolute certainty without parsing the data, but the usual case would be handled by holding onto the last-N bytes or so. Enough to fit the EOF and perhaps a little more for paranoia's sake.

That's not to say that I'm missing the problem. When (not "if", "when") the user feeds data past a CopyData(EOF), it's going to get interesting.

[Thinking about the logic necessary to handle such a case and avoid network buffer deadlock...]
I would think the least invasive way to handle it would be to set the CommandComplete and ReadyForQuery messages aside when they are received if CopyDone hasn't been sent, continue the COPY operation as usual until it is shutdown, send CopyDone and, finally, "reinstate" CommandComplete and RFQ as if they were just received.. I don't think that really accommodates for CopyFail as the status in RFQ will need to be "adjusted" to match what was actually done..? Well, I'm not sure you would need to worry about NoticeResponse after a premature CommandComplete as INSERTs are no longer happening. ugh.

+1 for a fix.

Not directly regarding your patch, but while the discussion is in the general area.
I think it would be wise to throw an error when non-empty CopyData messages are received after CopyData(EOF). Chances are that the user is making a mistake and should be notified of it.

cheers, jwp