Re: batch inserts are "slow"

From: "David Parker" <dparker(at)tazznetworks(dot)com>
To: "Christopher Kings-Lynne" <chriskl(at)familyhealth(dot)com(dot)au>, <tim(at)se(dot)linux(dot)org>
Cc: <pgsql-performance(at)postgresql(dot)org>
Subject: Re: batch inserts are "slow"
Date: 2005-05-02 15:53:33
Message-ID: 07FDEE0ED7455A48AC42AC2070EDFF7C67EF1F@corpsrv2.tazznetworks.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

We ran into the need to use COPY, but our application is also in Java.
We wrote a JNI bridge to a C++ routine that uses the libpq library to do
the COPY. The coding is a little bit weird, but not too complicated -
the biggest pain in the neck is probably getting it into your build
system.

Here's the Java tutorial on JNI:
http://java.sun.com/docs/books/tutorial/native1.1/concepts/index.html

Hope that helps!

- DAP

>-----Original Message-----
>From: pgsql-performance-owner(at)postgresql(dot)org
>[mailto:pgsql-performance-owner(at)postgresql(dot)org] On Behalf Of
>Christopher Kings-Lynne
>Sent: Monday, May 02, 2005 11:11 AM
>To: tim(at)se(dot)linux(dot)org
>Cc: pgsql-performance(at)postgresql(dot)org
>Subject: Re: [PERFORM] batch inserts are "slow"
>
>> conn.setAutoCommit(false);
>> pst = conn.prepareStatement("INSERT INTO tmp (...) VALUES
>(?,?)"); for
>> (int i = 0; i < len; i++) {
>> pst.setInt(0, 2);
>> pst.setString(1, "xxx");
>> pst.addBatch();
>> }
>> pst.executeBatch();
>> conn.commit();
>>
>> This snip takes 1.3 secs in postgresql. How can I lower that?
>
>You're batching them as one transaction, and using a prepared
>query both of which are good. I guess the next step for a
>great performance improvement is to use the COPY command.
>However, you'd have to find out how to access that via Java.
>
>I have a nasty suspicion that the release JDBC driver doesn't
>support it and you may have to apply a patch.
>
>Ask on pgsql-jdbc(at)postgresql(dot)org perhaps.
>
>Chris
>
>---------------------------(end of
>broadcast)---------------------------
>TIP 1: subscribe and unsubscribe commands go to
>majordomo(at)postgresql(dot)org
>

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Gurmeet Manku 2005-05-02 16:14:00 Citation for "Bad n_distinct estimation; hacks suggested?"
Previous Message Tom Lane 2005-05-02 15:29:59 Re: batch inserts are "slow"