Skip site navigation (1) Skip section navigation (2)

Synchronisation problems in COPY IN

From: Kim Bisgaard <kim+pg(at)alleroedderne(dot)adsl(dot)dk>
To: pgsql-jdbc(at)postgresql(dot)org
Subject: Synchronisation problems in COPY IN
Date: 2010-09-13 12:13:24
Message-ID: 4C8E1564.80405@alleroedderne.adsl.dk (view raw or flat)
Thread:
Lists: pgsql-jdbc
Hi,

We are occasionally experiencing problems with our JDBC implementation 
of COPY IN.

We have cut our program down to the attached example.

Guessing from the error messages it looks like jdbc is overwriting the 
buffer sent to the server before it is done. We are writing the data in 
CSV format, with integer(serial),date(now),float format, so it is easy 
to see if the buffer is garbled. Here is an example:
INFO: ERROR: extra data after last expected column
   Where: COPY temporary_wind_speed_50_m_3, line 175: 
"60000179,2010-09-07 00:00:00,60001043,2010-09-07 
00:00:00,10.032084733490484"
org.postgresql.util.PSQLException: ERROR: extra data after last expected 
column
   Where: COPY temporary_wind_speed_50_m_3, line 175: 
"60000179,2010-09-07 00:00:00,60001043,2010-09-07 
00:00:00,10.032084733490484"
         at 
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
         at 
org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:929)
         at 
org.postgresql.core.v3.QueryExecutorImpl.writeToCopy(QueryExecutorImpl.java:838)
         at 
org.postgresql.core.v3.CopyInImpl.writeToCopy(CopyInImpl.java:53)
         at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:179)
         at org.postgresql.copy.CopyManager.copyIn(CopyManager.java:161)
         at dk.dmi.appl.evejrfeeder.EvejrDBCopy.run(Unknown Source)
         at java.lang.Thread.run(Thread.java:595)


It thus looks like a synchronisation problem (locking problem), but we 
are using Piped(Input/Output)Stream, which must be properly 
synchronised, and CopyManager.CopyIn, where there is nowhere to add 
synchronisation.

To us it looks like a problem in CopyManager.CopyIn, or alternatively we 
are not doing something we where expected to do?

We use PostgreSQL 8.4, and JDBC 8.4-702.

Thanks in advance!

Kim


Attachment: EvejrDBCopy.java
Description: text/x-java (3.4 KB)

Responses

pgsql-jdbc by date

Next:From: Kris JurkaDate: 2010-09-13 14:17:14
Subject: Re: Synchronisation problems in COPY IN
Previous:From: Craig RingerDate: 2010-09-12 01:20:09
Subject: Re: Send a command to postgres and close the program

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group