Skip site navigation (1) Skip section navigation (2)

Re: Why "copy ... from stdio" does not return immediately when reading invalid data?

From: John R Pierce <pierce(at)hogranch(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: Why "copy ... from stdio" does not return immediately when reading invalid data?
Date: 2011-02-02 19:06:01
Message-ID: 4D49AB19.4020502@hogranch.com (view raw or flat)
Thread:
Lists: pgsql-generalpgsql-hackers
On 02/02/11 10:20 AM, Nicolas Grilly wrote:
> Is the copy protocol (aka PQputCopyData and PQputCopyEnd) designed to 
> send gigabytes of data with just one "copy ... from stdio" query, and 
> is there a way to be notified of a potential error before calling 
> PQputCopyEnd? Or do I have to send my data in small chunks (for 
> example batch of 10000 rows), issue a PQputCopyEnd, check for errors, 
> and continue with the next chunk?

I would batch the data, maybe 1000 lines or even 100 lines at a time if 
these errors are at all frequent.  put the errored batches in an 
exception list or something so you can sort them out later.



In response to

Responses

pgsql-hackers by date

Next:From: vaibhavkaushal123Date: 2011-02-02 19:25:12
Subject: Elephants
Previous:From: Greg SmithDate: 2011-02-02 18:47:21
Subject: Re: [HACKERS] Slow count(*) again...

pgsql-general by date

Next:From: William BrutonDate: 2011-02-02 19:08:19
Subject: upgrade
Previous:From: Carlos MennensDate: 2011-02-02 18:32:41
Subject: Database Design Question

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group