Skip site navigation (1) Skip section navigation (2)

Re: {Spam} 转发: [ADMIN] Copy cmd error

From: Dimitri Fontaine <dfontaine(at)hi-media(dot)com>
To: pgsql-admin(at)postgresql(dot)org, qinghuamail-postgresqlfans(at)yahoo(dot)com
Subject: Re: {Spam} 转发: [ADMIN] Copy cmd error
Date: 2007-08-28 08:03:57
Message-ID: (view raw or whole thread)
Lists: pgsql-admin

Le mardi 28 août 2007, qinghuamail-postgresqlfans(at)yahoo(dot)com a écrit :
> hi, after reading the source of pgloader,i have a idea,use perl to create
> the flat file, and, when error,reject the row,seek the next line,continue
> to copy.

That's pretty much what pgloader already does for you: if COPY errors out, 
pgloader split the data in two groups and try to COPY again each of them, 
until the dichotomy isolates the one row causing COPY the error.
Then the given row is appended to the configured reject_data file.

See those options for details:


When issuing COPY PostgreSQL commands, pgloader will not make a single big 
COPY attempt, but copy copy_every lines at a time.
This parameter is optionnal and defaults to 10000.

 In case of errors processing input data, a human readable log per rejected 
input data line is produced into the reject_log file. 

 In case of errors processing input data, the rejected input line is appended 
to the reject_data file.

Maybe this feature should be better highlighted from the DESCRIPTION section 
of the manpage (which is also the website).


In response to

pgsql-admin by date

Next:From: Jordi MuletDate: 2007-08-28 10:01:46
Subject: No restore of BLOB data (permissions?)
Previous:From: Kevin KempterDate: 2007-08-28 03:52:56
Subject: Re: how to tell what tables have been vacuumed ?

Privacy Policy | About PostgreSQL
Copyright © 1996-2015 The PostgreSQL Global Development Group