Re: [JDBC] Re: Too much data received?

Steve Wampler wrote:
>
> Steve Wampler wrote:
> >
> ...
> > Too much data was received.
> > at org.postgresql.PG_Stream.ReceiveString(PG_Stream.java:306)
> > at org.postgresql.Connection.ExecSQL(Connection.java:532)
> > at org.postgresql.jdbc2.Statement.execute(Statement.java:294)
> > at org.postgresql.jdbc2.Statement.executeUpdate(Statement.java:78)
> > at OCS.database.DbServer.execute(DbServer.java:271)
> >
> > I seem to remember discussion on a limit in the jdbc driver that might
> > be causing this, but couldn't find it in the psql-jdbc mail archives.
> >
> > Am I likely correct in this being the problem? Is there a fix in the
> > works?
>
> I took a look at the code for the jdbc driver and see how to increase the
> size of the receive string buffer. (I think that there is no limit on
> the size of a query/command, just on results - is that correct?)
>
> Is any one looking at removing the limit entirely? I can (and have)
> bumped the limit up, but wonder if it might not be better to just avoid
> the problem entirely (if the back end can handle sending an absurdly
> long result, why shouldn't the front end accept it?). Since the
> data is coming from the backend (presumably a comparitively slow
> operation), it doesn't seem that any performance hit would be
> particularly noticable (if well designed) and could likely be restricted
> to those cases where the results are absurdly long...

I'm not sure if anyone else is interested, but I have a version of
the PostgreSQL jdbc driver that places no restrictions on the length
of a received string (except in places where there was already
a low (e.g. 4096) limit). So you can insert/retrieve an arbitrary-length
text field to/from the backend. If the length is <= 65535 bytes then
performance should be the same as in the current version. There is
a performance hit for longer strings (briefly, the byte buffer grows by
50% each time it fills), but that's just the price for using absurdly
long text fields (it would be simple to parameterize the driver
to tune this performance, but I haven't done so to keep the API the
same.)

It could use some more rigorous testing, but it works for my purposes
just fine.

Is anyone interested in looking at this? If so, someone will have to
tell me how to generate context diffs against the CVS repository.
(There aren't many changes to the code.)

Thanks!

(The problem originated when storying XMLized java objects into the
database - the jdbc driver couldn't handle the long strings produced
by the JSX java-to/from-xml package.)

--
Steve Wampler- SOLIS Project, National Solar Observatory
swampler(at)noao(dot)edu