RE: Slowdown problem when writing 1.7million records

From: "Stephen Livesey" <ste(at)exact3ex(dot)co(dot)uk>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: <pgsql-general(at)postgresql(dot)org>
Subject: RE: Slowdown problem when writing 1.7million records
Date: 2001-02-28 09:20:03
Message-ID: GDENKIKOJLMLLEIOOCCICELLCDAA.ste@exact3ex.co.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

> No, it's not. Do you have any triggers or rules on this table that
> you haven't shown us? How about other tables referencing this one
> as foreign keys? (Probably not, if you're running an identical test
> on MySQL, but I just want to be sure that I'm not missing something.)

I have no triggers or rules. I have only created 1 table and their are no
foreign keys.

>
> How exactly are you writing the records?

First I read the data from a 'Powerflex' file and hold this in a record set.
pfxstmt = pfxconn.createStatement();
pfxrs = pfxstmt.executeQuery("SELECT * from expafh");

I then perform a loop which writes the data to my 'Postgresql' file as
follows:
stmt = conn.createStatement();
while (pfxrs.next()) {
cmd = "INSERT INTO expafh VALUES ";
cmd = cmd +
"('"+pfxrs.getString(2)+"',"+pfxrs.getString(3)+",'"+pfxrs.getString(4)+"','
"+pfxrs.getString(5)+"')";
stmt.executeUpdate(cmd);
}

>
> I have a suspicion that the slowdown must be on the client side (perhaps
> some inefficiency in the JDBC code?) but that's only a guess at this
> point.
>

I have used identical code for all of my testing, the only changes being
which drivers I use to access the data.

Thanks
Stephen Livesey

Legal Disclaimer:
Internet communications are not secure and therefore Exact Abacus does
not accept legal responsibility for the contents of this message. Any views
or opinions presented are solely those of the author and do not necessarily
represent those of Exact Abacus unless otherwise specifically stated.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Enrico Mangano 2001-02-28 09:22:45 Improve performance
Previous Message Enrico Mangano 2001-02-28 09:09:11