Skip site navigation (1) Skip section navigation (2)

Workarounds for getBinaryStream returning ByteArrayInputStream on bytea

From: Александър Шопов <lists(at)kambanaria(dot)org>
To: pgsql-jdbc(at)postgresql(dot)org
Subject: Workarounds for getBinaryStream returning ByteArrayInputStream on bytea
Date: 2010-11-24 20:50:46
Message-ID: 1290631846.3659.9.camel@dalgonosko (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-jdbc
Hi everyone,
I have a table containing file contents in bytea columns.
The functionality I am trying to achieve is having a result set
containing such columns, iterating over them and streaming them while
zipping them.
The problem is that I get ByteArrayInputStream from
ResultSet.getBinaryStream.
Thus iterating over many rows, each containing more than 10MB of data
smashes the heap. In peak times I will have several such processes. 
I am using postgresql-8.4-702.jdbc3.jar against a PG 8.4.5 installation.
I looked at the current source of driver.
Jdbc3ResultSet extends AbstractJdbc3ResultSet extends
AbstractJdbc2ResultSet which is the place that provides implementation
for  getBinaryStream which returns ByteArrayInputStream on bytea
columns, and BlobInputStream on blob columns. On skimming it seems that
BlobInputStream does indeed stream the bytes instead of reading them in
memory (chunks for reads are 4k).
So what am I options? Refactor the DB schema to use blobs rather than
bytea? Is it impossible to have bytea read in chunks?
Kind regards:
al_shopov



Responses

pgsql-hackers by date

Next:From: Andres FreundDate: 2010-11-24 20:53:20
Subject: Re: profiling connection overhead
Previous:From: Robert HaasDate: 2010-11-24 20:47:32
Subject: Re: profiling connection overhead

pgsql-jdbc by date

Next:From: Radosław SmoguraDate: 2010-11-24 22:04:49
Subject: Re: Workarounds for getBinaryStream returning ByteArrayInputStream on bytea
Previous:From: Maciek SakrejdaDate: 2010-11-24 18:25:27
Subject: Re: [JDBC] JDBC and Binary protocol error, for some statements

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group