Re: slow inserts

From: Jodi Kanter <jkanter(at)virginia(dot)edu>
To: Postgres Admin List <pgsql-admin(at)postgresql(dot)org>
Subject: Re: slow inserts
Date: 2002-03-20 20:04:44
Message-ID: 009d01c1d04a$7ac4a6f0$de138f80@virginia.edu
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

None of the data is actually committed to the database until the scripts
complete so I believe that autocommit is turned off.

----- Original Message -----
From: "Nick Fankhauser" <nickf(at)ontko(dot)com>
To: "Jodi Kanter" <jkanter(at)virginia(dot)edu>; "Postgres Admin List"
<pgsql-admin(at)postgresql(dot)org>
Sent: Wednesday, March 20, 2002 2:42 PM
Subject: Re: [ADMIN] slow inserts

> Jodi-
>
> Have you tried turning autocommit off & doing a single commit after the
> load?
>
> -NickF
>
> -----Original Message-----
> From: pgsql-admin-owner(at)postgresql(dot)org
> [mailto:pgsql-admin-owner(at)postgresql(dot)org]On Behalf Of Jodi Kanter
> Sent: Wednesday, March 20, 2002 2:19 PM
> To: Postgres Admin List
> Subject: [ADMIN] slow inserts
>
>
> I am currently using a Perl data loader that was set up to load data to
> three particular tables. The majority of this data is entered into one
> table. The scripts obtain the data by parsing it out of an Excel
> spreadsheet. My loads seem to take a very long time. Each file only has
> about 12,000 rows, yet it takes close to 25 minutes to load one file. I
have
> placed some debugging syntax in the code and it seems that the extra time
if
> related to postgres as I had originally thought it may have to do with the
> parsing of the Excel file.
>
> I have tried turning off FSYNC but there is no difference in load time. I
> also tried removing the indexes from the table in which most of the data
is
> loaded but no luck. By the way, is there anyway to confirm that the FSYNC
> option was turned off correctly? a way to view status or something?
>
> Any thoughts on what might be going on? another performance tuning trick
> that I have not thought of?
>
> Thanks for your help,
> Jodi Kanter
>
> _______________________________
> Jodi L Kanter
> BioInformatics Database Administrator
> University of Virginia
> (434) 924-2846
> jkanter(at)virginia(dot)edu
>
>
>
>
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
>

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Christopher Smith 2002-03-20 22:26:04 large table support 32,000,000 rows
Previous Message Nick Fankhauser 2002-03-20 19:42:03 Re: slow inserts