Re: Interest in allowing caller to push binary data rather than having it pulled?

Tom,

I'd love to see this!

Dave Cramer

davec(at)postgresintl(dot)com
www.postgresintl.com

On 23 March 2017 at 04:24, Tom Dunstan <pgsql(at)tomd(dot)cc> wrote:

> Hi all
>
> I hit an interesting case today. It’s a bit of a limitation in the JDBC
> interface, so any support would have to be a proprietary interface.
>
> Basically I have one or more byte buffers that I’d like to stream into a
> BYTEA at the server (using a plain INSERT statement). In my case I’ve got
> Netty ByteBuf objects, but it could be anything.
>
> What are my current options? JDBC basically gives me
> PreparedStatement.setBytes() and PreparedStatement.setBinaryStream().
>
> PreparedStatement.setBytes() involves copying all the data, potentially
> multiple large buffers, into a large buffer of exactly the correct size.
> The reason to use ByteBufs in the first place was to pool our use of large
> buffers so that we don’t blow out our heap - this completely kills any hope
> of that.
>
> PreparedStatement.setBinaryStream() is more flexible, but under the hood
> we’re just pulling stuff into an intermediary 8k buffer and then writing it
> out to the socket. This is OK from a heap management perspective, but still
> has some unnecessary copying.
>
> What I’d really like to do would be to provide an object that the driver
> could interrogate for a length and then provide an OutputStream to write
> to. The interface would look something like:
>
> interface ByteStreamWriter {
> int getLength();
> void writeTo(OutputStream stream);
> }
>
> The provided output stream would be a very thin wrapper around the socket
> output stream just ensuring that we don’t write too many bytes out.
>
> Usage would look thusly:
>
> myPreparedStatement.setObject(n, new MyByteStreamWriter(myByteBuf),
> Types.VARBINARY);
>
> And the user could write whatever adapter they wanted around their data.
>
> There’s an existing StreamWrapper class in the codebase, but it just
> provides an InputStream when asked. It could be adjusted to use the above
> interface for consistency though.
>
> Thoughts? I’d be happy to code up a PR if there’s interest.
>
> Cheers
>
> Tom
>
>
>
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc
>