Re: COPY support in JDBC driver?

Hi Mark,

Batching statements in JDBC will probably get you up to 1000 operations/sec
(reading from disk, slow database) or 10000 operations/sec (reading from
memory and/or fast database). I have not yet used the JDBC COPY patch, but
it might be what you want, if you need higher speeds. I had always
associated the COPY command with properly formatted input files, but I have
read all the patch needs is an InputStream, so it might be exactly what you
need.

I am planning to package pgBee into a library, so that it may be used from
other java programs or become part of a workflow, but it's always going to
be file based. Then again, when you say 'immense amount of data' and
'periodically' you make me worry about JVM coping with your memory
requirements.

cheers,

Michael

On Thu, Oct 16, 2008 at 12:14 AM, <mark_addleman(at)bigfoot(dot)com> wrote:

>
> Hi, Michael -
>
> Thanks for the batch pointer. We are currently using batches with
> varying sizes but still aren't getting the performance we'd like.
>
> As for including the COPY facility in the JDBC driver versus doing
> bulk loads from a command line utility: Our online application
> receives an immense amount of data periodically and must store that
> data into Postgres before the next bulk arrives. It makes sense to
> process and store this data entirely within the confines of a JVM and
> within a transaction thus COPY support in the JDBC driver is very
> helpful for us.
>
>