Timely reporting of COPY errors

From: Martijn van Oosterhout <kleptog(at)svana(dot)org>
To: Postgresql <pgsql-hackers(at)postgresql(dot)org>
Subject: Timely reporting of COPY errors
Date: 2008-04-16 20:29:07
Message-ID: 20080416202907.GA26340@svana.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


I notice that while doing bulk-loads that any errors detected by the
backend arn't noticed by libpq until right at the end. Is this
intentional? Looking at the code we have this comment in putCopyData:

* Process any NOTICE or NOTIFY messages that might be pending in the
* input buffer. Since the server might generate many notices during the
* COPY, we want to clean those out reasonably promptly to prevent
* indefinite expansion of the input buffer. (Note: the actual read of
* input data into the input buffer happens down inside pqSendSome, but
* it's not authorized to get rid of the data again.)

Except that pqSendSome won't try reading anything until it has a
problem writing. Since the backend will consume copy data indefinitly,
the error message sits in the kernel buffers until the end.

Is there anything that can be done? I've tried putting in
PQconsumeInput in places but it doesn't appear to help.

Any ideas?
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.


Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2008-04-16 20:36:13 Re: How to submit a patch
Previous Message Joshua D. Drake 2008-04-16 20:21:10 Re: How to submit a patch