Loading data into Postgres

From: Przemyslaw Bak <przemol(at)st3(dot)makro(dot)com(dot)pl>
To: pgsql-general(at)postgresql(dot)org
Subject: Loading data into Postgres
Date: 1998-05-31 15:22:37
Message-ID: Pine.SV4.3.96.980531164247.23497A-100000@STORE3
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,

every night my script (using perl) will load about 50.000 rows of
data into Postgres. I know two ways:

In first version of my script I used method by inserting, eg:
---------- cut ----------
#!/bin/perl

use Pg;

open (MyData, "...");
while (<MyData>) {
chop;
...
&insert_cust_sales ($NDate, $CustNo, $ArtGrp, $Qty, $Amount);
}
close MyData;

sub insert_cust_sales {
my $SQL_QUERY = "insert into cust_sales values ('$_[0]',$_[1],$_[2],$_[3],$_[4],$_[5]);";
my $conn = Pg::connectdb ("dbname = store3");
my $result = $conn->exec ( $SQL_QUERY );
}
---------- cut ----------
but it is very slow and my disk works very havy.
So I created another solution, look below:
---------- cut ----------
#!/bin/perl

# Open connection to Postgres
open (PSQL, "| psql store3") || die "Can not open connection to Postgres\n";
print PSQL " copy cust_sales from stdin using delimiters '|';";

# Read data from file
open (MyData, "...");
while (<MyData>) {
s/ //g; # remove \t
s/\s+//g; # ... and spaces
print PSQL,$_,"\n";
}
print PSQL "\\.\n";
close PSQL;
close MyData;
--------- cut ----------
but while running this script I receive message "resetting connection".
Why ? What's wrong with this script ? I have seen in sources
of postgres that this message is when copy failed, but why ?

przemol

Browse pgsql-general by date

  From Date Subject
Next Message Marin D 1998-05-31 15:36:33 libpq.so missing
Previous Message Marin D 1998-05-31 15:10:43 no shared libs