NOTICE messages during COPY OUT

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-hackers(at)postgreSQL(dot)org, pgsql-interfaces(at)postgreSQL(dot)org
Subject: NOTICE messages during COPY OUT
Date: 2008-01-12 23:32:11
Message-ID: 23759.1200180731@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-interfaces

It was pointed out to me off-list that pg_dump fails if a user-defined
datatype's output routine is silly enough to emit NOTICE messages:
the rest of the table data simply disappears without any warning.

What is happening is that libpq's pqGetCopyData3() drops out of COPY_OUT
state upon seeing a NoticeResponse message. The rest of the data
arrives OK but is silently discarded by libpq because it's no longer in
COPY_OUT state.

Our protocol specification is ambiguous about this case: in
http://developer.postgresql.org/pgdocs/postgres/protocol-flow.html#PROTOCOL-COPY
it is stated that

: The frontend should treat receipt of ErrorResponse (or indeed any
: message type other than CopyData or CopyDone) as terminating the
: copy-out mode.

which seems to legitimize libpq's behavior. But the very next section
states that

: ... frontends should always be prepared to accept and display
: NoticeResponse messages, even when the connection is nominally idle.

and

: Good practice is to be able to accept NotificationResponse at any
: point in the protocol.

and there is similar verbiage about ParameterStatus.

So there are basically two ways we could respond to this: decide that
libpq is broken, and fix it; or decide that we should suppress
non-CopyData messages during COPY OUT. I dislike the second
alternative, because one of the motivations for dividing COPY data
into messages in the V3 protocol was specifically to allow other stuff
to be interpersed into the message stream. (We had to suppress Notice
messages during COPY OUT in the old protocol, but that restriction was
supposedly gotten rid of in V3; the backend thinks so anyway.)

An unknown (at this point) is what other client-side code might need to
be fixed if we decide it's a client-side bug.

One argument in favor of fixing it on the client side is that we
customarily advise people to use the newer pg_dump during a version
upgrade; so a client-side fix would be more likely to be effective
during an upgrade.

An argument against fixing it on the client side is that it's at least
conceivable that some apps would fail to cope with NoticeProcessor
callbacks during COPY OUT. However that seems like it's just an
application bug anyway.

This seems like a fairly serious bug, since it could lead to dumps that
are silently missing some data. The fact that we've not seen it
reported before, though it's been this way since 7.4, suggests that
the actual number of places that could emit NOTICEs during COPY OUT is
pretty low --- but I have a bad feeling that it's not zero, even
without considering debugging printouts in user-written data types.
It seems at least theoretically possible for ParameterStatus reports
to be emitted, too.

Comments?

regards, tom lane

Browse pgsql-hackers by date

  From Date Subject
Next Message Jonah H. Harris 2008-01-13 00:03:20 Re: Postgresql Materialized views
Previous Message Mark Mielke 2008-01-12 22:31:32 Re: Postgresql Materialized views

Browse pgsql-interfaces by date

  From Date Subject
Next Message Greg Sabino Mullane 2008-01-16 04:21:07 DBD::Pg 2.0.0 release candidate available for testing
Previous Message Tom Lane 2008-01-11 01:09:58 Re: URGENET: ALTER statement exectuation hangs when it is executed multiple time on a table