Re: Synchronisation problems in COPY IN

There is some moderately bogus behavior with respect to closing a
connection while doing a COPY (see list archives). I wonder if that
could be involved?

If memory serves, the issue was that a Terminate message (sent on
Connection.close() to indicate that the client wants to end the
protocol connection) is *not* synchronized relative to COPY protocol
operations. This means that *theoretically*, your Terminate message
could be shimmed right into the middle of a CopyData message. In
practice, I've only ever seen an error from the server complaining
about an unexpected message type (Terminate) during COPY, since the
client is *supposed* to send CopyDone or CopyFail before a Terminate
while a COPY is in effect. Still, I think it might be possible to get
message contents munged.

I think your code *might* be susceptible to that, since you kick off a
bunch of COPY operations in separate threads and then call
Connection.close() in the main thread, without waiting for the COPY
threads to finish if I'm reading that right--should that if
(t.isAlive) {} be something while (t.isAlive()) {}?

Can you capture (e.g., tcpdump) the data stream that gets sent to the
server? It'd be interesting to see if Terminate actually is getting
munged into the COPY data or if we're looking at another issue
entirely.

For what it's worth, I investigated a fix, but (1) it's a pain in the
ass to get the logic right without Java 5's synchronization primitives
(in fact, if we do this again, it might be worth just implementing a
ReadWriteLock unless JDBC is planning to drop 1.4 support) and (2)
based on preliminary testing, there was an unacceptable performance
regression in doing the locking right. I can try to revisit the patch
(or dig it out and post it as is), but we may need to rethink this
entirely.

First of all, though, it'd be nice to figure out if this is actually
the problem affecting you. If the problem I described is still just
theoretical, I wouldn't worry about it. Since it involves the client
closing the Connection while having active COPY operations in other
threads, this is wonky *application* behavior any (well, almost any)
way you look at it.

---
Maciek Sakrejda | System Architect | Truviso

1065 E. Hillsdale Blvd., Suite 215
Foster City, CA 94404
(650) 242-3500 Main
www.truviso.com

On Mon, Sep 13, 2010 at 7:17 AM, Kris Jurka <books(at)ejurka(dot)com> wrote:
>
>
> On Mon, 13 Sep 2010, Kim Bisgaard wrote:
>
>> We are occasionally experiencing problems with our JDBC implementation of
>> COPY IN.
>>
>> We have cut our program down to the attached example.
>
> Do you get the failure you've shown from this example?  I get another
> failure from the connection being close before the last copy thread can
> complete its write operation.  This seems like an expected failure, but I
> can't reproduce yours.
>
> org.postgresql.util.PSQLException: Database connection failed when canceling
> copy operation
>        at
> org.postgresql.core.v3.QueryExecutorImpl.cancelCopy(QueryExecutorImpl.java:796)
>        at
> org.postgresql.core.v3.CopyOperationImpl.cancelCopy(CopyOperationImpl.java:32)
>        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:150)
>        at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:126)
>        at EvejrDBCopy.run(EvejrDBCopy.java:34)
>
>> We use PostgreSQL 8.4, and JDBC 8.4-702.
>>
>
> Your stacktrace shows that you are using 8.4-701, not 702.  702 had some
> fixes to the copy code when using a Reader, but your code shows you are
> using an InputStream, so those changes shouldn't be relevent.
>
> Kris Jurka
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc
>