Re: BLOB performance test FYI

From: Dave Cramer <Dave(at)micro-automation(dot)net>
To: Joe Shevland <jshevland(at)j-elite(dot)com>
Cc: Anders Bengtsson <ndrsbngtssn(at)yahoo(dot)se>, "pgsql-jdbc(at)postgresql(dot)org" <pgsql-jdbc(at)postgresql(dot)org>
Subject: Re: BLOB performance test FYI
Date: 2002-04-18 01:10:22
Message-ID: 1019092223.2084.24.camel@inspiron.cramers
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-jdbc

Joe, Anders,

Please keep this discussion online. The more people we have looking at a
possible solution, the better.

Dave
On Wed, 2002-04-17 at 20:37, Joe Shevland wrote:
> > > > I find that
> > > > many users of bytea are storing gifs and html pages for web
> > sites which
> > > > generally are small enough such that bytea works fine.
> > >
> > > Well, I disagree... even a few kilobytes has shown to be substandard in
> > > storing.
> >
> > Some of these problems aren't really about the datatypes themselves, but
> > how the driver handles them.
>
> Thats a good point; my issues have purely been through the JDBC driver interaction.
>
> > Instead of streaming large data into the
> > backend, the client reads them into memory before sending them.
> > It seems the LO parts of the driver has a different solution, that
> > streams the data directly.
> > It would take some major reworking of the JDBC driver to fix this. I've
> > looked at it, and concluded that it is possible, but that it was too big
> > for me to fix.
>
> I've got time (read: resigned a few weeks ago and am sick of hop-flavoured water ;) and wouldn't mind doing it; I just needed an indication that its possible. Would you mind giving me some hints about what you looked at offline?
>
> I figured that the differences in the types and how the backend handles them precludes actually trying to do anything to optimise the behaviour in the client interface. For example, can I use the socket connection when performing an update operation to stream binary data to the backend, or does it expect the whole statement to be character data sent at the one time?
>
> What about other client interfaces and bytea types, how does the TCP/IP stuff go? C, Perl et al would still have problems loading binary data into memory too I would have thought unless they stream the data.
>
> I'll add some notes re: this discussion to http://www.j-elite.com/pgprimer sometime today or tomorrow in the BLOB section, then put aside some time to put some real content into the rest of the site.
>
> Cheers,
> Joe
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> message can get through to the mailing list cleanly
>
>

In response to

Responses

Browse pgsql-jdbc by date

  From Date Subject
Next Message Barry Lind 2002-04-18 01:32:15 Re: BLOB performance test FYI
Previous Message Joe Shevland 2002-04-18 00:37:37 Re: BLOB performance test FYI