Re: [HACKERS] libpq

Chris <chris(at)bitmead(dot)com> writes:
> What I have in mind is that a new API PQexecStream() doesn't retrieve
> the results. The tuples are then read back one by one with
> PQnextObject().

OK, but how does this interact with asynchrononous retrieval? It
should be possible to run it in a nonblocking (select-waiting) mode.

> /* like PQexec, but streams the results */
> PGresult *PQexecStream(PGconn *conn, const char *query)
> /* retrieve the next object from a PGresult */
> PGobject *PQnextObject(PGconn *conn)
> /* get value from an object/tuple */
> char *PQgetObjectValue(const PGobject *res, int field_num)
> /* free tuple when done */
> void PQclearObject(PGobject *obj)

There are two other big gaps here, which is that you haven't specified
how you represent (a) errors and (b) end of query result. I assume you
intend the initial PQexecStream call to wait for the first tuple to come
back, so *most* sorts of errors will be reported at that point, but
you have to be able to cope with errors reported later on too.

Rather than inventing a new PGobject struct type, I'd suggest returning
the partial results as PGresults. This has a couple of benefits:
* easy representation of an error encountered midway (you just return
an error PGresult).
* it's no big trick to "batch" retrieval, ie, return 10 or 100 tuples
at a time, if that happens to prove useful.
* each tuple batch could carry its own tuple description, which is
something you will need if you want to go anywhere with that
polymorphic-results idea.
* end-of-query could be represented as a PGresult with zero tuples.
(This would leave a null-pointer result open for use in the nonblock
case, to indicate "haven't got a response yet".)
* no need for an entire new set of API functions to query PGobjects.

BTW, an earlier proposal for this same sort of thing didn't see it
as an entirely new operating mode, but just a "limit" option added
to a variant of PQexec: the limit says "return no more than N tuples
per PQresult".

> Oh yeah, can I fix the COPY protocol while I'm at it to conform more to
> the other types of messages?

I looked at that before, and while COPY is certainly ugly as sin, it's
not clear that it's worth creating cross-version compatibility problems
to fix it. I'm inclined to leave it alone until such time as we
undertake a really massive protocol change (moving to CORBA, say).

> BTW, what is this PQ thing? Does it stand for postquel? Are we ever
> going to dump that?

Yes, and no. We aren't going to break existing app code by indulging
in cosmetic renaming of API names. Moreover we have to have *some*
prefix to minimize the probability of global-symbol conflicts with apps
and other libraries, so that one's as good as any.

To the extent that there is any system in the names in libpq (which I
admit ain't much), it's
PQfoo --- exported public-API routine
pqfoo --- internal routine not meant for apps to call, but must
be global symbol because it is called cross-module
PGfoo --- type name, enum const, etc
I'd suggest sticking to those conventions in any new code you write.

regards, tom lane