Skip site navigation (1) Skip section navigation (2)

Re: slow inserts

From: "Peter Darley" <pdarley(at)kinesis-cem(dot)com>
To: "Jodi Kanter" <jkanter(at)virginia(dot)edu>,"Postgres Admin List" <pgsql-admin(at)postgresql(dot)org>
Subject: Re: slow inserts
Date: 2002-03-20 19:31:29
Message-ID: NNEAICKPNOGDBHNCEDCPKEFNCGAA.pdarley@kinesis-cem.com (view raw or flat)
Thread:
Lists: pgsql-admin
Jodi,
    I believe DBI defaults to autocommitting queries, which slows it down.
If you aren't specifically turning off the autocommit, I'd suggest it, which
would make all the inserts go in one transaction which makes it much faster.
You'd do something like:
    DBI->connect("dbi:Pg:dbname=dbname;host=hostname", 'user', 'password',
{AutoCommit => 0})
Thanks,
Peter Darley
  -----Original Message-----
  From: pgsql-admin-owner(at)postgresql(dot)org
[mailto:pgsql-admin-owner(at)postgresql(dot)org]On Behalf Of Jodi Kanter
  Sent: Wednesday, March 20, 2002 11:19 AM
  To: Postgres Admin List
  Subject: [ADMIN] slow inserts


  I am currently using a Perl data loader that was set up to load data to
three particular tables. The majority of this data is entered into one
table. The scripts obtain the data by parsing it out of an Excel
spreadsheet. My loads seem to take a very long time. Each file only has
about 12,000 rows, yet it takes close to 25 minutes to load one file. I have
placed some debugging syntax in the code and it seems that the extra time if
related to postgres as I had originally thought it may have to do with the
parsing of the Excel file.

  I have tried turning off FSYNC but there is no difference in load time. I
also tried removing the indexes from the table in which most of the data is
loaded but no luck. By the way, is there anyway to confirm that the FSYNC
option was turned off correctly? a way to view status or something?

  Any thoughts on what might be going on? another performance tuning trick
that I have not thought of?

  Thanks for your help,
  Jodi Kanter

  _______________________________
  Jodi L Kanter
  BioInformatics Database Administrator
  University of Virginia
  (434) 924-2846
  jkanter(at)virginia(dot)edu









In response to

pgsql-admin by date

Next:From: Bruce MomjianDate: 2002-03-20 19:38:49
Subject: Re: problems with dbf-files
Previous:From: Jodi KanterDate: 2002-03-20 19:18:41
Subject: slow inserts

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group