Re: Stuff for 2.4.1

On Sunday, March 27, 2011 11:25:20 am Daniele Varrazzo wrote:
> On Sun, Mar 27, 2011 at 6:23 PM, David Blewett <david(at)dawninglight(dot)net> wrote:
> > On Sun, Mar 27, 2011 at 9:46 AM, Daniele Varrazzo
> >
> > <daniele(dot)varrazzo(at)gmail(dot)com> wrote:
> >> On Sun, Mar 27, 2011 at 12:51 PM, Harald Armin Massa
> >>
> >> <harald(at)2ndquadrant(dot)com> wrote:
> >>> Are you really sure that psycopg2 should go the road of having own
> >>> parsers in addition to libpq-s routines? As much as I am happy about
> >>> the robustness when having other libpqs, and about the performance
> >>> benefit, as much I fear to have some new areas for possible bugs -
> >>> especially security-relevant things like SQL-injections.
> >>
> >> The will to stick as much as possible to the libpq functions has been
> >> the reason I had not written the above parser before (releasing
> >> 2.4.0). Unfortunately the bytea problem has proven trickier to handle
> >> for many psycopg users. I've changed my mind as I think psycopg has
> >> the responsibility to provide a set of feature in a robust way, and if
> >> the libpq is just not reliable for bytea parsing (for me the hex
> >> format should have been backported to the the client libraries of the
> >> previous versions) I think we have to provide a solution, not just to
> >> propagate the problem.
> >
> > I think I agree with Harald here. In my opinion, this shouldn't be
> > done at the driver level.
>
> No? At which level do you think this should be done?
>
> > There never has been a guarantee from the
> > database side that applications compiled against older libpq will be
> > able to communicate with newer versions. Emulating this in the driver
> > only propagates this mis-conception. What has been the problem in the
> > past?
>
> There have been several examples in the not so long story of this
> mailing list, other were in the previous ml. Here's a couple. You can
> google yourself to discover that GnuMed, Drupal and who knows how many
> not public projects have been bitten by the transition to hex.

This is fixed by:
bytea_output = escape

Major upgrades are just that, major. Failure to read the docs is on the user. I
know, I got bit by the change in behavior of pl/python with regards to 't' or
'f'.

>
> http://archives.postgresql.org/psycopg/2011-02/msg00020.php
> http://archives.postgresql.org/pgsql-general/2010-10/msg00146.php
> (from Harald, interesting)
>
> > Maybe the documentation should be improved so that people are
> > sure to build against the appropriate version of libpq for the version
> > of the server they intend to communicate with?
>
> The documentation does a pretty good job in describing the problem.
> Unfortunately not everybody is willing/able/allowed to install an
> up-to-date client library. PG 9 is not so widespread on the clients
> yet (e.g. Natty still deploys 8.4, so you will have to wait at least
> Oct 2011 for a libpq 9 on the most widespread linux distro).

I am not entirely against your argument, but if you are going to make the above
statement at least compare apples to apples. In Natty the released version of
Psycopg2 is 2.2.1 , so 2.4.1 is not an option by your rules. Also, it is
possible for people to run a Pg 9.0 on newer distributions:
https://launchpad.net/~pitti/+archive/postgresql

>
> Half of Psycopg job is to convert the textual representation of
> postgres data type into python object. The libpq provides no
> assistance in doing that for any other format than bytea. There is no
> PQunescapeDateTime whatsoever: should we ask Python users to parse the
> dates from string themselves? Not to mention composite types, hstore
> and other delicacies, for which we provide parsers to Python objects.
> The bytea is in a sense "blessed" for having a dedicated unescape
> function, but if using it causes more trouble than it solves I think a
> less problematic parser is a good answer.

Like I said I am not entirely against what you are trying to do. I am concerned
though that hiding a implementation detail does not really serve the user.
Sooner or later they will have to come to grips with the change in behavior,
whether via Psycopg or some other interface.

>
> -- Daniele

--
Adrian Klaver
adrian(dot)klaver(at)gmail(dot)com