Skip site navigation (1) Skip section navigation (2)

Re: copy_from does not stop reading after an error

From: Marko Kreen <markokr(at)gmail(dot)com>
To: Nicolas Grilly <nicolas(at)gardentechno(dot)com>
Cc: Federico Di Gregorio <federico(dot)digregorio(at)dndg(dot)it>, psycopg(at)postgresql(dot)org
Subject: Re: copy_from does not stop reading after an error
Date: 2011-02-08 12:30:31
Message-ID: AANLkTi=6fJ3vWfNQdxZRog5fzvi+hS9U2H-FxNfpr-G7@mail.gmail.com (view raw or flat)
Thread:
Lists: psycopg
On Mon, Feb 7, 2011 at 5:06 PM, Nicolas Grilly <nicolas(at)gardentechno(dot)com> wrote:
> On Tue, Feb 1, 2011 at 14:18, Marko Kreen <markokr(at)gmail(dot)com> wrote:
>>
>> No, the error message should arrive immediately.  But it may be deficiency
>> of
>> libpq that you cannot aquire it before ending the copy.
>
> I have analyzed the PostgreSQL protocol using Wireshark (an open source
> packet analyzer), and I confirm that the PostgreSQL backend, while doing a
> COPY ... FROM STDIN, reports errors as soon as possible (especially errors
> related to invalid data).
> Therefore, the "late" reporting of errors while doing a COPY ... FROM STDIN
> is not a limitation of the underlying protocol; it is a limitation (or a
> design choice) of the libpq library.
> It looks like this is a well known issue because it is listed on the todo
> list:
> http://wiki.postgresql.org/wiki/Todo#COPY
> And was discussed before:
> http://archives.postgresql.org/pgsql-hackers/2008-04/msg01169.php
>
>>
>> I'm not sure about that actually.  It should be possible to call
>> select() & PQconsumeInput
>> between copy calls, thus also PQgetResult to get the error.
>
> I have tried that, and many other things, and I have carefully read libpq
> source code, and my understanding is that it is impossible with the current
> implementation of libpq.
> It is impossible because the function pqParseInput3 (file fe-protocol3.c)
> does not parse error responses while the connection is in PGASYNC_COPY_IN
> state. We have to call PQputCopyEnd to switch the connection to PGASYNC_BUSY
> and start error messages parsing.
> Can Marko or someone else confirm this analysis? Any idea to improve that?
> Regards,
> Nicolas Grilly
> PS : Considering that both psycopg2 and psql are built on top of libpq, this
> a quite logical they behave the same regarding late reporting of copy
> errors, contrary to pg8000 that could easily read, parse and report the
> errors early.

Yeah, seems your analysis is right and it's libpq (design?) bug,
so no way to work around it.

-- 
marko

In response to

Responses

psycopg by date

Next:From: Nicolas GrillyDate: 2011-02-08 12:38:20
Subject: Re: copy_from does not stop reading after an error
Previous:From: Nicolas GrillyDate: 2011-02-07 15:06:19
Subject: Re: copy_from does not stop reading after an error

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group