libpq and Binary Data Formats

First of all, apologies if this was not meant to be a feedback/wishlist
mailing list.

Binary formats in libpq has been (probably) a long
issue (refer to the listings below) and I want to express my hope that the
next revision of PostgreSQL would have better support for binary data types
in libpq. I am in no doubt that those binary vs. text debates sprouted
because of PostgreSQL's (or rather libpq's) ambiguity when it comes to
binary data support. One instance is the documentation itself: it didn't
really say (correct me if I'm wrong) that binary data is poorly/not
supported and that textual data is preferred. Moreover, those ambiguities
are only cleared up in mailing lists/irc/forums which make it seem that the
arguments for text data is just an excuse to not have proper support for
binary data (e.x. C:"Elephant doesn't support Hammer!" P: "You don't really
need Hammer (we don't support it yet), you can do it with Screwdriver.").
This is not meant to be a binary vs. text post so I'll reserve my comments
for them. Nevertheless, they each have their own advantages and
disadvantages especially when it comes to strongly typed languages that
neither shouldn't be ignored.

I am well-aware of the problems associated with binary formats and
backward/forward compatibility:
http://archives.postgresql.org/pgsql-hackers/1999-08/msg00374.php but
nevertheless, that shouldn't stop PostgreSQL/libpq's hardworking developers
from coming up with a solution. The earling link showed the interest of
using CORBA to handle PostgreSQL objects but I belive that it's an overkill
and would like to propose using ASN.1 instead. However, what's important is
not really the binary/text representation. If we look again the the list
below, not everyone need binary formats just for speed and efficiency,
rather, they need it to be able to easily manipulate data. In other words,
the interfaces to extract data is also important.

Best wishes,
Wil

NOTES/History of Posts:

1: "Query regarding PostgreSQL date/time binary format for libpq" <
http://archives.postgresql.org/pgsql-interfaces/2007-01/msg00040.php> One of
the many (clueless) individuals who wants to get the binary format of the
date/time struct (I know that there's a way to do this be converting the
time to epoch using extract(epoch from time) to convert it to somthing akin
to time_t)
2. "Bytea network traffic: binary vs text result format" <
http://archives.postgresql.org/pgsql-interfaces/2007-06/msg00000.php> One of
the many Binary vs. Text debates.
3. "How do you convert PostgreSQL internal binary field to C datatypes" <
http://archives.postgresql.org/pgsql-interfaces/2007-05/msg00046.php> An
individual disgruntled because of the "half baked C API" of PostgreSQL.
Although he may be wrong in some or many aspects, he has a point with
regards to the binary format support. Moreover, he is probably one of the
many individuals who are disappointed on PostgreSQL because of this.
4. "Array handling in libpq" <
http://archives.postgresql.org/pgsql-interfaces/2007-01/msg00027.php> One of
the common scenarios for the "need" of a binary format (or rather, a better
interface): arrays. Also, the reply of this is one of the many/redundant
assurances that the overhead of text is minimal.
5. "libpq PQexecParams and arrays" <
http://archives.postgresql.org/pgsql-interfaces/2006-06/msg00008.php>
Another one of those array issues. This time, the poster/s have expressed
that the documentation for binary formats is "poorly documented :-("
6. "PQgetvalue failed to return column value for non-text data in binary
format" <
http://archives.postgresql.org/pgsql-interfaces/2007-05/msg00045.php>
Another issue about binary formats paired with the assurance (again) that
the overhead of using text is minimal.
--
(<_<)(>_>)(>_<)(<.<)(>.>)(>.<)
Life is too short for dial-up.