Skip site navigation (1) Skip section navigation (2)

Re: Stuff for 2.4.1

From: Adrian Klaver <adrian(dot)klaver(at)gmail(dot)com>
To: psycopg(at)postgresql(dot)org
Cc: Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com>, David Blewett <david(at)dawninglight(dot)net>, harald(at)2ndquadrant(dot)com
Subject: Re: Stuff for 2.4.1
Date: 2011-03-27 19:01:09
Message-ID: (view raw, whole thread or download thread mbox)
Lists: psycopg
On Sunday, March 27, 2011 11:25:20 am Daniele Varrazzo wrote:
> On Sun, Mar 27, 2011 at 6:23 PM, David Blewett <david(at)dawninglight(dot)net> wrote:
> > On Sun, Mar 27, 2011 at 9:46 AM, Daniele Varrazzo
> > 
> > <daniele(dot)varrazzo(at)gmail(dot)com> wrote:
> >> On Sun, Mar 27, 2011 at 12:51 PM, Harald Armin Massa
> >> 
> >> <harald(at)2ndquadrant(dot)com> wrote:
> >>> Are you really sure that psycopg2 should go the road of having own
> >>> parsers in addition to libpq-s routines? As much as I am happy about
> >>> the robustness when having other libpqs, and about the performance
> >>> benefit, as much I fear to have some new areas for possible bugs -
> >>> especially security-relevant things like SQL-injections.
> >> 
> >> The will to stick as much as possible to the libpq functions has been
> >> the reason I had not written the above parser before (releasing
> >> 2.4.0). Unfortunately the bytea problem has proven trickier to handle
> >> for many psycopg users. I've changed my mind as I think psycopg has
> >> the responsibility to provide a set of feature in a robust way, and if
> >> the libpq is just not reliable for bytea parsing (for me the hex
> >> format should have been backported to the the client libraries of the
> >> previous versions) I think we have to provide a solution, not just to
> >> propagate the problem.
> > 
> > I think I agree with Harald here. In my opinion, this shouldn't be
> > done at the driver level.
> No? At which level do you think this should be done?
> > There never has been a guarantee from the
> > database side that applications compiled against older libpq will be
> > able to communicate with newer versions. Emulating this in the driver
> > only propagates this mis-conception. What has been the problem in the
> > past?
> There have been several examples in the not so long story of this
> mailing list, other were in the previous ml. Here's a couple. You can
> google yourself to discover that GnuMed, Drupal and who knows how many
> not public projects have been bitten by the transition to hex.

This is fixed by:
bytea_output = escape

Major upgrades are just that, major. Failure to read the docs is on the user. I 
know, I got bit by the change in behavior of pl/python with regards to 't' or 

> (from Harald, interesting)
> > Maybe the documentation should be improved so that people are
> > sure to build against the appropriate version of libpq for the version
> > of the server they intend to communicate with?
> The documentation does a pretty good job in describing the problem.
> Unfortunately not everybody is willing/able/allowed to install an
> up-to-date client library. PG 9 is not so widespread on the clients
> yet (e.g. Natty still deploys 8.4, so you will have to wait at least
> Oct 2011 for a libpq 9 on the most widespread linux distro).

I am not entirely against your argument, but if you are going to make the above 
statement at least compare apples to apples. In Natty the released version of 
Psycopg2 is 2.2.1 , so 2.4.1 is not an option by your rules. Also, it is 
possible for people to run a Pg 9.0 on newer distributions:

> Half of Psycopg job is to convert the textual representation of
> postgres data type into python object. The libpq provides no
> assistance in doing that for any other format than bytea. There is no
> PQunescapeDateTime whatsoever: should we ask Python users to parse the
> dates from string themselves? Not to mention composite types, hstore
> and other delicacies, for which we provide parsers to Python objects.
> The bytea is in a sense "blessed" for having a dedicated unescape
> function, but if using it causes more trouble than it solves I think a
> less problematic parser is a good answer.

Like I said I am not entirely against what you are trying to do. I am concerned 
though that hiding a implementation detail does not really serve the user. 
Sooner or later they will have to come to grips with the change in behavior, 
whether via Psycopg or some other interface. 

> -- Daniele

Adrian Klaver

In response to


psycopg by date

Next:From: Karsten HilbertDate: 2011-03-27 19:18:26
Subject: Re: Stuff for 2.4.1
Previous:From: Harald Armin MassaDate: 2011-03-27 18:56:52
Subject: Re: Stuff for 2.4.1

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group