Skip site navigation (1) Skip section navigation (2)

Re: Workarounds for getBinaryStream returning ByteArrayInputStream on bytea

From: Radosław Smogura <rsmogura(at)softperience(dot)eu>
To: Александър Шопов <lists(at)kambanaria(dot)org>
Cc: <pgsql-jdbc(at)postgresql(dot)org>
Subject: Re: Workarounds for getBinaryStream returning ByteArrayInputStream on bytea
Date: 2010-11-25 09:43:29
Message-ID: 857fe9c145d2d7cbf99d27123613bcec@smogura-softworks.eu (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-jdbc
On Thu, 25 Nov 2010 00:53:31 +0200, Александър Шопов
<lists(at)kambanaria(dot)org>
wrote:
> В 16:04 -0600 на 24.11.2010 (ср), Radosław Smogura написа:
>> I see only two possibilities
>> 1. Decrease fetch size, e.g. to 1.
> Even if I do, bytea is potentially 1GB. Plus peaks in usage can still
> smash the heap.
bytea is like varchar, and it's transmitted to client at all, even if you
don't want to read it, it is somewhere on heap.

> So refactoring to BLOBs is perhaps the only way out.
Here is other solution
http://archives.postgresql.org/pgsql-jdbc/2007-08/msg00078.php
You can write simple stream to do such reads on demand, select everything
without bytea column..., but probably blobs will be better.

> Will the JDBC driver always present bytea InputStream as
> ByteArrayInputStream? No plans to change that? (even if there are, I
> will still have to refactor meanwhile).
As above, the content is on heap, much more when you read you transform
this content and you creates new array (so heap 2x), maybe some chunked  on
demand transformation will be better, but this is driver specific... or you
need to wait when I end binary JDBC and this will be in main release...

> Perhaps this behaviour should be better communicated to DB schema
> designers.
> It seems to me from the Npgsql2.0.11 readme.txt that reading in chunks
> is provided for .Net.
At a glance it looks that chunk reading means reading all network response
at all not bytea.

> Is there need to perhaps make patches for this in the jdbc driver?
> Kind regards:
> al_shopov

----------
Radosław Smogura
http://www.softperience.eu

In response to

pgsql-hackers by date

Next:From: Koichi SuzukiDate: 2010-11-25 09:45:45
Subject: Re: Horizontal Write Scaling
Previous:From: Markus WannerDate: 2010-11-25 09:03:53
Subject: Re: Horizontal Write Scaling

pgsql-jdbc by date

Next:From: Maciek SakrejdaDate: 2010-11-25 09:56:02
Subject: Re: [JDBC] JDBC and Binary protocol error, for some statements
Previous:From: Maciek SakrejdaDate: 2010-11-25 07:25:14
Subject: Re: [JDBC] JDBC and Binary protocol error, for some statements

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group