From: | Przemyslaw Bak <przemol(at)st3(dot)makro(dot)com(dot)pl> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Loading data into Postgres |
Date: | 1998-05-31 15:22:37 |
Message-ID: | Pine.SV4.3.96.980531164247.23497A-100000@STORE3 |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Hi,
every night my script (using perl) will load about 50.000 rows of
data into Postgres. I know two ways:
In first version of my script I used method by inserting, eg:
---------- cut ----------
#!/bin/perl
use Pg;
open (MyData, "...");
while (<MyData>) {
chop;
...
&insert_cust_sales ($NDate, $CustNo, $ArtGrp, $Qty, $Amount);
}
close MyData;
sub insert_cust_sales {
my $SQL_QUERY = "insert into cust_sales values ('$_[0]',$_[1],$_[2],$_[3],$_[4],$_[5]);";
my $conn = Pg::connectdb ("dbname = store3");
my $result = $conn->exec ( $SQL_QUERY );
}
---------- cut ----------
but it is very slow and my disk works very havy.
So I created another solution, look below:
---------- cut ----------
#!/bin/perl
# Open connection to Postgres
open (PSQL, "| psql store3") || die "Can not open connection to Postgres\n";
print PSQL " copy cust_sales from stdin using delimiters '|';";
# Read data from file
open (MyData, "...");
while (<MyData>) {
s/ //g; # remove \t
s/\s+//g; # ... and spaces
print PSQL,$_,"\n";
}
print PSQL "\\.\n";
close PSQL;
close MyData;
--------- cut ----------
but while running this script I receive message "resetting connection".
Why ? What's wrong with this script ? I have seen in sources
of postgres that this message is when copy failed, but why ?
przemol
From | Date | Subject | |
---|---|---|---|
Next Message | Marin D | 1998-05-31 15:36:33 | libpq.so missing |
Previous Message | Marin D | 1998-05-31 15:10:43 | no shared libs |