COPY problem.

I am seeing the following error:

ERROR: end-of-copy marker corrupt
CONTEXT: COPY moreover_documents_y2010m06, line 49327

What is that? The code that produces the error above is extremely simple:

sub do_copy
{
$dbh->do("COPY $table FROM STDIN WITH NULL AS E'\e'");
while (<>)
{
die($dbh->errstr . "\n") if ($dbh->pg_putcopydata($_) != 1);
}
$dbh->pg_putcopyend();
$dbh->commit();
}

I am copying the textual data, provided by the following routine:

my $FILE = "|write_table -t $dest -n $ncols";
$FILE .= " -c" if defined($copy);
open(STDOUT, $FILE) || die("Cannot open output:$!\n");

while (my @row = $sth->fetchrow_array())
{
map { $_ = "\e" if !defined($_); s/[\t\n]+/ /g; } @row;
my $line = join $sep, @row;
print "$line\n";
--$nrows;
last if ($nrows == 0);
}

The $sth is a SQL statement handle which selects from another database.
There are several millions of rows and it is hard to figure out which
one has failed. I am trying with a normal SQL insert, but that is much
slower.

--

Mladen Gogala
Sr. Oracle DBA
1500 Broadway
New York, NY 10036
(212) 329-5251
http://www.vmsinfo.com
The Leader in Integrated Media Intelligence Solutions