Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] Trouble with COPY IN

From: James William Pye <lists(at)jwp(dot)name>
To: Kris Jurka <books(at)ejurka(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org, Matthew Wakeling <matthew(at)flymine(dot)org>, Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>, Maciek Sakrejda <msakrejda(at)truviso(dot)com>, Samuel Gendler <sgendler(at)ideasculptor(dot)com>, pgsql-jdbc(at)postgresql(dot)org
Subject: Re: [HACKERS] Trouble with COPY IN
Date: 2010-07-28 16:31:32
Message-ID: 41218459-EDA4-4005-B791-2392991150AC@jwp.name (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-jdbc
On Jul 25, 2010, at 8:01 AM, Kris Jurka wrote:
> The JDBC driver reads server messages for multiple reasons.

> One of them is indeed to do early failure detection.

That's high quality. =)

> Another is to pickup NoticeResponse messages to avoid a network buffer deadlock.

That's a good catch. I don't think psql/restore would often run into this as when COPY IN is in play, it's normally restoring a database. However, with JDBC, I imagine COPY would more often be used to do bulk loading into live tables that may very well cause a NOTICE. [Well, I reference psql/libpq because I don't recall it recognizing failure during COPY IN in the past, so I assume it's not receiving any data in that state.]

hrm, I suppose a lazy way around that problem would be to suspend all client messages(client_min_messages) during COPY IN. Tho, I guess one would still have to contend with NotificationResponse, and ParameterStatus..

> So this is possible to work around driver side by peeking into the network stream and delaying processing of the end of copy until the driver agrees that the copy is done, but

I don't think you would have to peek in. If the interface were to always hold onto the last message or last n-bytes submitted to be sent, it would be able to send the possible CopyData(EOF) and CopyDone once the COPY operation (at the interface level) is closed/shutdown/terminated. Granted, this is dependent on CopyData(EOF) not being in the middle of regular CopyData, but I gather that that would end in an ErrorResponse anyways.

> I still maintain that this is a server bug. It is not OK for the server to assume that the client is done and move on, the client must tell the server what it wants done.


I'm a bit torn here. While it would seem to be either a bug in the spec or a bug in the server, I'm inclined to call it a wart in the server's implementation of the spec.

I don't see the fix as being dangerous, but I imagine an implementor would want to have the workaround in place regardless. I certainly would.

I'd be in favor of seeing this fixed in 9.x, and the documentation updated to warn implementors about the wart in the older versions.. That is, I don't see any reason why we can't get rid of this unsightly thing considering the workarounds would still work with a wart-free server.

cheers, jwp

In response to

Responses

pgsql-hackers by date

Next:From: Joshua D. DrakeDate: 2010-07-28 16:33:42
Subject: Re: Toward a column reorder solution
Previous:From: David E. WheelerDate: 2010-07-28 16:30:06
Subject: Re: Toward a column reorder solution

pgsql-jdbc by date

Next:From: Kris JurkaDate: 2010-07-28 16:53:49
Subject: Re: [HACKERS] Trouble with COPY IN
Previous:From: dmpDate: 2010-07-28 16:02:32
Subject: Re: getMetaData().getTables() behaviour with JDBC3 8.3 & 8.4 drivers on windows

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group