Skip site navigation (1) Skip section navigation (2)

Re: OutOfMemory when inserting stream of unknown length

From: Kris Jurka <books(at)ejurka(dot)com>
To: "Mikko T(dot)" <mtiihone(at)cc(dot)hut(dot)fi>
Cc: pgsql-jdbc(at)postgresql(dot)org
Subject: Re: OutOfMemory when inserting stream of unknown length
Date: 2004-08-19 18:42:39
Message-ID: Pine.BSO.4.56.0408191336180.14552@leary.csoft.net (view raw or flat)
Thread:
Lists: pgsql-jdbc

On Thu, 19 Aug 2004, Mikko T. wrote:

> 
> When writing a java stream of unknown length to a bytea column with the 
> prepared statement setBinaryStream the jdbc drivers throws always an 
> OutOfMemoryException.
> 
> I'm calling the function with following parameters as I do not know the real 
> length of the stream:
> 
> stm.setBinaryStream(1, stream, Interger.MAX_VALUE);
> 
> and it seems that somewhere along the way the jdbc driver wrongly
> assumes that the stream contains gigabytes of data and a memory
> allocation fails.

This is true of the stable driver branch, it will try to allocate a huge 
amount of memory.  The development driver has the ability to stream data 
to the backend, but unfortunately for you requires an exact length 
argument.  The problem is that the frontend/backend protocol messages are 
preceded with a length parameter.  So the options are to read the whole 
stream to determine the length (which requires huge amounts of memory to 
store the whole stream), or take the provided length argument at face 
value and stream data to the server hoping the user was right.  In the 
streaming case it's ok if the length argument is less than the length of 
the stream because then we can just stop reading, but not if the stream 
itself is short.

Kris Jurka


In response to

pgsql-jdbc by date

Next:From: Oliver JowettDate: 2004-08-19 21:27:40
Subject: Re: OutOfMemory when inserting stream of unknown length
Previous:From: Mikko T.Date: 2004-08-19 10:44:06
Subject: OutOfMemory when inserting stream of unknown length

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group