Re: Incremental results from libpq

On Nov 9, 2005, at 2:09 PM, Scott Lamb wrote:
> Cool. I think I'll get my own interface hashed out in my kludgey
> way, then look at the broader need if it's a success.

I've done some of this. I've got the start of a new Python<-
>PostgreSQL interface at <http://www.slamb.org/svn/repos/trunk/
projects/apypg/>.

Currently, it just "peeks" at connection->result after polling, grabs
a bunch of results and handles them, then empties the resultset
kludgily. (If it has a different idea of PGRESULT_DATA_BLOCKSIZE than
libpq, it'll probably crash.)

Aside from the nastiness of playing with libpq's internal data
structures, it doesn't handle the case Tom mentioned particularly
well. It will handle an undetermined fraction of the rows before the
error, then error out. I think it should handle _all_ the rows it
gets before the error consistently.

> My first idea, though, is to add a callback interface - "got the
> RowDescription", "got a DataRow" - and make the storage of stuff
> tuples in PGresult optional. (Maybe pqAddTuple would just be the
> default callback.)

Actually, I'm not sure about this anymore. There's a complication -
if I want to do something fancy while handling a row - a Python
exception, C++ exception, thread cancellation - I don't think there's
any good way to do that in a callback structure.

Another complication is the limitation of one active resultset. libpq
needs to be extended to support cursors other than the unnamed one
('') in order to handle other statements as it iterates over an
incremental result set.

--
Scott Lamb <http://www.slamb.org/>