Re: [Fwd: Re: [Fwd: Re: Problems with truncated BLOB]]

Mike,

I am really at a loss to explain what is going on here. I just
resurected a 7.1.2 database and ran the test and it worked fine (for
both the bytea, and LargeObject versions using the 7.2 driver).

thanks,
--Barry

Mike Cannon-Brookes wrote:

> Barry,
>
> I'm definitely just using the new driver, here's my classpath:
>
> /usr/java/jdk1.3.1/lib/tools.jar:.:/opt/orion/lib/jdbc7.2dev-1.2.jar
>
> (downloaded the jdbc jar from jdbc.postgresql.org this morning)
>
> Might I suggest it's something to do with bytea support from 7.1 to
> 7.2b3? ;)
>
> -mike
>
> On Tue, 2001-11-27 at 13:13, Barry Lind wrote:
>
>>Mike,
>>
>>When testing with the latest 7.2 drivers are you sure you are using the
>>latest drivers and not picking up an older version of the driver
>>somewhere else in your classpath? I can't reproduce the bug using the
>>latest drivers but can reproduce with 7.1 drivers (see below for my
>>successful results)
>>
>>thanks,
>>--Barry
>>
>>[blind(at)barry work]$ CLASSPATH=.:./postgresql.jar; export CLASSPATH
>>[blind(at)barry work]$ java BlobTest postgresql.jar
>>Connected to database system.
>>
>>file.name:postgresql.jarfile.length:116228
>>File.length(int): 116228
>>Trying to write: postgresql.jartest
>>We have a result!
>>bytes written: 116228
>>[blind(at)barry work]$
>>
>>The only thing different in my environment is that I am running 7.2b2 on
>>the server instead of 7.1.3. Otherwise I am also using RedHat 7.2 and
>>2.4 kernel.
>>
>>
>>
>>
>>
>>
>>Mike Cannon-Brookes wrote:
>>
>>
>>>Barry,
>>>
>>>This program perfectly illustrates the bug for me (using both latest
>>>drivers from website - uploaded 25/11 - and 7.1 drivers).
>>>
>>>Here's some sample output:
>>>
>>>[mike(at)frodo tmp]$ java BlobTest postgresql.jar
>>>Connected to database system.
>>>
>>>file.name: postgresql.jar file.length: 95059
>>>File.length(int): 95059
>>>Trying to write: postgresql.jartest
>>>We have a result!
>>>We have a result!
>>>bytes written: 476
>>>
>>>(used the postgresql.jar as a test binary file ;)).
>>>
>>>It seems to work ok with text files, but not with binary files. I'd say
>>>there's definitely a bug here somewhere.
>>>
>>>Postgres DB version is 7.1.3, Linux 2.4, RH 7.2
>>>
>>>-mike
>>>
>>>On Tue, 2001-11-27 at 08:18, Barry Lind wrote:
>>>
>>>
>>>>Mihael,
>>>>
>>>>I'll include the attachments this time.
>>>>
>>>>thanks,
>>>>--Barry
>>>>
>>>>-------- Original Message --------
>>>>Subject: Re: [Fwd: Re: Problems with truncated BLOB]
>>>>Date: Mon, 26 Nov 2001 12:27:58 -0800
>>>>From: Barry Lind <barry(at)xythos(dot)com>
>>>>To: miqster(at)gmx(dot)net
>>>>CC: pgsql-jdbc(at)postgresql(dot)org
>>>>References: <3C028151(dot)8020301(at)xythos(dot)com>
>>>>
>>>>Mihael,
>>>>
>>>>I have been able to reproduce your problem with the 7.1 drivers, but not
>>>>with the 7.2 drivers. It appears that this is bug is already fixed in
>>>>current sources.
>>>>
>>>>Note however that the functionality around get/setBinaryStream() has
>>>>changed in 7.2. In 7.2 the default behavior is that binary data is
>>>>assumed to be in a column of type bytea. So you would create a table
>>>>as: create table blobs (filename text, data bytea) instead of using the
>>>>OID datatype as in 7.1 (create table blobs (filename text, data OID)).
>>>>For backward compatibility you can revert the driver back to the old
>>>>behavior of using LargeObjects (i.e. OIDs) by passing the compatible=7.1
>>>>parameter to the connection. (This can be done by addint
>>>>'?compatible=7.1' to the end of the jdbc connection url.
>>>>
>>>>Attached are modified versions of the test case you sent that work for
>>>>me using the 7.2 drivers (but do show the bug using the 7.1 drivers).
>>>>The first (BlobTest.java) uses the new functionality and requires that
>>>>the table 'blobs' have a column named 'data' of type 'bytea'. The
>>>>second (BlobTest2.java) uses the backward compatible mode and requires
>>>>that the table 'blobs2' have a column named 'data' of type 'oid'.
>>>>
>>>>thanks,
>>>>--Barry
>>>>
>>>>Barry Lind wrote:
>>>>
>>>>
>>>>>Forwarding to the jdbc list in case someone else has time to look into
>>>>>this bug before I get around to it.
>>>>>
>>>>>thanks,
>>>>>--Barry
>>>>>
>>>>>-------- Original Message --------
>>>>>Subject: Re: [JDBC] Problems with truncated BLOB
>>>>>Date: 26 Nov 2001 17:37:05 +0100
>>>>>From: Mihael Vrbanec <miqster(at)gmx(dot)net>
>>>>>To: Barry Lind <barry(at)xythos(dot)com>
>>>>>References: <1006737949(dot)367(dot)2(dot)camel(at)Caprice>
>>>>>
>>>><3C01A2AA(dot)2030902(at)xythos(dot)com>
>>>>
>>>>>Hi Barry,
>>>>>
>>>>> > Could you send a test case that demonstrates the problem? Ideally a
>>>>> > sample java program that will upload one of the problem files, and
>>>>>
>>>>then
>>>>
>>>>> > another program that tries to download the file showing the
>>>>>
>>>>problem you
>>>>
>>>>> > have having.
>>>>>
>>>>>I have attached a sample program that does both. It uses a table called
>>>>>blobs with two columns (filename, data). I experience the problem with
>>>>>all binary files I have tried (.doc, .png, .bmp, .tar.gz).
>>>>>HTML, and .txt do not have these problems.
>>>>>
>>>>> > Also, which version of the jdbc driver are you using? (the
>>>>>
>>>>version that
>>>>
>>>>> > came with 7.1, or something newer)
>>>>>
>>>>>The jdbc-driver is the one shipped with 7.1.2/3 (i compiled the whole
>>>>>thing from source).
>>>>>
>>>>>many thanx in advance
>>>>>
>>>>>bxe Miq
>>>>>
>>>>>
>>>>>------------------------------------------------------------------------
>>>>>
>>>>>import java.io.*;
>>>>>import java.util.*;
>>>>>import java.sql.*;
>>>>>
>>>>>
>>>>>public class BlobTest {
>>>>>
>>>>> private Connection con = null;
>>>>>
>>>>> public BlobTest() {
>>>>> try {
>>>>> Class.forName("org.postgresql.Driver");
>>>>> con = DriverManager.getConnection(
>>>>> "jdbc:postgresql:repository", "candle", "");
>>>>> if (con != null) {
>>>>> System.out.println("Connected to database system.\n");
>>>>> }
>>>>> } catch (SQLException e) {
>>>>> System.out.println(e.getMessage());
>>>>> System.out.println("Could not connect to database
>>>>>
>>>>system.\n");
>>>>
>>>>> } catch (ClassNotFoundException e) {
>>>>> System.out.println("Class not found...:-(\n");
>>>>> }
>>>>> }
>>>>>
>>>>> private void store(String filename) {
>>>>> PreparedStatement ps = null;
>>>>> ResultSet r = null;
>>>>> File file = new File(filename);
>>>>> System.out.println("file.name:" + file.getName() +
>>>>>
>>>>"file.length:"+file.length());
>>>>
>>>>> try {
>>>>> FileInputStream fis = new FileInputStream(file);
>>>>> con.setAutoCommit(false);
>>>>> ps = con.prepareStatement(
>>>>> "INSERT INTO blobs VALUES (?, ?)");
>>>>> ps.setString(1, filename);
>>>>> System.out.println("File.length(int): " + (int)
>>>>>
>>>>file.length());
>>>>
>>>>> ps.setBinaryStream(2, fis, (int) file.length());
>>>>> ps.executeUpdate();
>>>>> ps.close();
>>>>> fis.close();
>>>>> con.commit();
>>>>> } catch (SQLException sqle) {
>>>>> System.err.println("Store content: " + sqle.getMessage());
>>>>> } catch (IOException ioe) {
>>>>> }
>>>>> }
>>>>>
>>>>> private void retrieve(String filename) {
>>>>> Statement s = null;
>>>>> ResultSet r = null;
>>>>> int byteSum = 0;
>>>>> int bytesRead = 0;
>>>>> byte[] buffer = new byte[8 * 1924];
>>>>> try {
>>>>> System.out.println("Trying to write: " +filename + "test");
>>>>> FileOutputStream out = new FileOutputStream(filename +
>>>>>
>>>>"test");
>>>>
>>>>> con.setAutoCommit(false);
>>>>> s = con.createStatement();
>>>>> r = s.executeQuery("SELECT data FROM blobs WHERE filename
>>>>>
>>>>= '"
>>>>
>>>>> + filename +"'");
>>>>> if (r != null) {
>>>>> while (r.next()) {
>>>>> System.out.println("We have a result!");
>>>>> InputStream is = r.getBinaryStream(1);
>>>>> while ((bytesRead = is.read(buffer)) != -1) {
>>>>> byteSum += bytesRead;
>>>>> out.write(buffer, 0, bytesRead);
>>>>> }
>>>>> is.close();
>>>>> }
>>>>> }
>>>>> out.close();
>>>>> System.out.println("bytes written: " + byteSum);
>>>>> con.commit();
>>>>> } catch (SQLException sqle) {
>>>>> System.err.println("Retrieve content: " + sqle.getMessage());
>>>>> } catch (Exception ioe) {
>>>>> System.err.println("Writing stuff: " + ioe.getMessage());
>>>>> }
>>>>> }
>>>>>
>>>>> public static void main(String[] args) {
>>>>> BlobTest bt = new BlobTest();
>>>>> bt.store(args[0]);
>>>>> bt.retrieve(args[0]);
>>>>> }
>>>>>
>>>>>}
>>>>>
>>>>>
>>>>>------------------------------------------------------------------------
>>>>>
>>>>>
>>>>>---------------------------(end of broadcast)---------------------------
>>>>>TIP 6: Have you searched our list archives?
>>>>>
>>>>>http://archives.postgresql.org
>>>>>
>>>>>
>>>>
>>>>----
>>>>
>>>>
>>>>
>>>>import java.io.*;
>>>>import java.util.*;
>>>>import java.sql.*;
>>>>
>>>>
>>>>public class BlobTest {
>>>>
>>>> private Connection con = null;
>>>>
>>>> public BlobTest() {
>>>> try {
>>>> Class.forName("org.postgresql.Driver");
>>>> con = DriverManager.getConnection(
>>>> "jdbc:postgresql://localhost:5432/files", "blind", "");
>>>> if (con != null) {
>>>> System.out.println("Connected to database system.\n");
>>>> }
>>>> } catch (SQLException e) {
>>>> System.out.println(e.getMessage());
>>>> System.out.println("Could not connect to database system.\n");
>>>> } catch (ClassNotFoundException e) {
>>>> System.out.println("Class not found...:-(\n");
>>>> }
>>>> }
>>>>
>>>> private void store(String filename) {
>>>> PreparedStatement ps = null;
>>>> ResultSet r = null;
>>>> File file = new File(filename);
>>>> System.out.println("file.name:" + file.getName() + "file.length:"+file.length());
>>>> try {
>>>> FileInputStream fis = new FileInputStream(file);
>>>> con.setAutoCommit(false);
>>>> ps = con.prepareStatement(
>>>> "INSERT INTO blobs VALUES (?, ?)");
>>>> ps.setString(1, filename);
>>>> System.out.println("File.length(int): " + (int) file.length());
>>>> ps.setBinaryStream(2, fis, (int) file.length());
>>>> ps.executeUpdate();
>>>> ps.close();
>>>> fis.close();
>>>> con.commit();
>>>> } catch (SQLException sqle) {
>>>> System.err.println("Store content: " + sqle.getMessage());
>>>> } catch (IOException ioe) {
>>>> }
>>>> }
>>>>
>>>> private void retrieve(String filename) {
>>>> Statement s = null;
>>>> ResultSet r = null;
>>>> int byteSum = 0;
>>>> int bytesRead = 0;
>>>> byte[] buffer = new byte[8 * 1924];
>>>> try {
>>>> System.out.println("Trying to write: " +filename + "test");
>>>> FileOutputStream out = new FileOutputStream(filename + "test");
>>>> con.setAutoCommit(false);
>>>> s = con.createStatement();
>>>> r = s.executeQuery("SELECT data FROM blobs WHERE filename = '"
>>>> + filename +"'");
>>>> if (r != null) {
>>>> while (r.next()) {
>>>> System.out.println("We have a result!");
>>>> InputStream is = r.getBinaryStream(1);
>>>> while ((bytesRead = is.read(buffer)) != -1) {
>>>> byteSum += bytesRead;
>>>> out.write(buffer, 0, bytesRead);
>>>> }
>>>> is.close();
>>>> }
>>>> }
>>>> out.close();
>>>> System.out.println("bytes written: " + byteSum);
>>>> con.commit();
>>>> } catch (SQLException sqle) {
>>>> System.err.println("Retrieve content: " + sqle.getMessage());
>>>> } catch (Exception ioe) {
>>>> System.err.println("Writing stuff: " + ioe.getMessage());
>>>> }
>>>> }
>>>>
>>>> public static void main(String[] args) {
>>>> BlobTest bt = new BlobTest();
>>>> bt.store(args[0]);
>>>> bt.retrieve(args[0]);
>>>> }
>>>>
>>>>}
>>>>----
>>>>
>>>>
>>>>
>>>>import java.io.*;
>>>>import java.util.*;
>>>>import java.sql.*;
>>>>
>>>>
>>>>public class BlobTest2 {
>>>>
>>>> private Connection con = null;
>>>>
>>>> public BlobTest2() {
>>>> try {
>>>> Class.forName("org.postgresql.Driver");
>>>> con = DriverManager.getConnection(
>>>> "jdbc:postgresql://localhost:5432/files?compatible=7.1", "blind", "");
>>>> if (con != null) {
>>>> System.out.println("Connected to database system.\n");
>>>> }
>>>> } catch (SQLException e) {
>>>> System.out.println(e.getMessage());
>>>> System.out.println("Could not connect to database system.\n");
>>>> } catch (ClassNotFoundException e) {
>>>> System.out.println("Class not found...:-(\n");
>>>> }
>>>> }
>>>>
>>>> private void store(String filename) {
>>>> PreparedStatement ps = null;
>>>> ResultSet r = null;
>>>> File file = new File(filename);
>>>> System.out.println("file.name:" + file.getName() + "file.length:"+file.length());
>>>> try {
>>>> FileInputStream fis = new FileInputStream(file);
>>>> con.setAutoCommit(false);
>>>> ps = con.prepareStatement(
>>>> "INSERT INTO blobs2 VALUES (?, ?)");
>>>> ps.setString(1, filename);
>>>> System.out.println("File.length(int): " + (int) file.length());
>>>> ps.setBinaryStream(2, fis, (int) file.length());
>>>> ps.executeUpdate();
>>>> ps.close();
>>>> fis.close();
>>>> con.commit();
>>>> } catch (SQLException sqle) {
>>>> System.err.println("Store content: " + sqle.getMessage());
>>>> } catch (IOException ioe) {
>>>> }
>>>> }
>>>>
>>>> private void retrieve(String filename) {
>>>> Statement s = null;
>>>> ResultSet r = null;
>>>> int byteSum = 0;
>>>> int bytesRead = 0;
>>>> byte[] buffer = new byte[8 * 1924];
>>>> try {
>>>> System.out.println("Trying to write: " +filename + "test");
>>>> FileOutputStream out = new FileOutputStream(filename + "test");
>>>> con.setAutoCommit(false);
>>>> s = con.createStatement();
>>>> r = s.executeQuery("SELECT data FROM blobs2 WHERE filename = '"
>>>> + filename +"'");
>>>> if (r != null) {
>>>> while (r.next()) {
>>>> System.out.println("We have a result!");
>>>> InputStream is = r.getBinaryStream(1);
>>>> while ((bytesRead = is.read(buffer)) != -1) {
>>>> byteSum += bytesRead;
>>>> out.write(buffer, 0, bytesRead);
>>>> }
>>>> is.close();
>>>> }
>>>> }
>>>> out.close();
>>>> System.out.println("bytes written: " + byteSum);
>>>> con.commit();
>>>> } catch (SQLException sqle) {
>>>> System.err.println("Retrieve content: " + sqle.getMessage());
>>>> } catch (Exception ioe) {
>>>> System.err.println("Writing stuff: " + ioe.getMessage());
>>>> }
>>>> }
>>>>
>>>> public static void main(String[] args) {
>>>> BlobTest2 bt = new BlobTest2();
>>>> bt.store(args[0]);
>>>> bt.retrieve(args[0]);
>>>> }
>>>>
>>>>}
>>>>----
>>>>
>>>>
>>>>
>>>>---------------------------(end of broadcast)---------------------------
>>>>TIP 2: you can get off all lists at once with the unregister command
>>>> (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
>>>>
>>>>
>>