non-blocking problems in libpq

From: Alfred Perlstein <bright(at)wintelcom(dot)net>
To: pgsql-interfaces(at)postgresql(dot)org
Subject: non-blocking problems in libpq
Date: 1999-12-11 04:47:52
Message-ID: Pine.BSF.4.21.9912101948400.4557-100000@fw.wintelcom.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-interfaces


I've just started getting some work going with postgresql and libpq,
one of the problems I'm having is the async COPY commands.

The problem seems to come from the buffering that's done automagically
by libpq. Here's the scenario:

PQsendQuery(connection, "copy mytable from stdin\n");
PQputline(conn, "3\thello world\t4.5\n");
PQputline(conn,"4\tgoodbye world\t7.11\n");
...
PQputline(conn,"\\.\n");

Just like on http://www.postgresql.org/docs/programmer/libpq-chapter3951.htm

here's the problem, a PQputline may 'half-buffer' a sent line, mainly
my "\\.\n", since instead of calling PQendcopy (which may block indefinetly)
I simply keep calling PQconsumeInput() and check PQisBusy().

Enless I call PQendcopy() (which may block indefinetly) the send buffer
may never actually be sent.

I'd like a way to flush the buffer associated with database connection
in a manner that won't block.

Is there a way to do this or a workaround?

Also there are several places where it appears that the routines in
libpq may block when sending data, it's related to the
fact that everytime you fill the PGconn's buffer it does a pqFlush()
which may block, is there a way around this?

I was thinking about modifying the library so that there is a function
to flush data explicitly, or that at least PQconsumeInput() will act
as a driver to flush the queued data.

Then again maybe it's my code that's not acting right. Can someone
give me hand here?

Also, I think i spotted a bug in the closePGconn() function, it calls
pqFlush() however pqFlush() may return EOF if there is a short write,
there may be outstanding data in the buffer.

Also, pqFlush doesn't deal with an EINTR error, which could be fixed
with a goto to restart the send() operation at the start of the
function.

I'm going to be checking the CVS logs to see if any of this is
addressed in the newer stuff, my apologies if it is, however, I'm
under considerable time constriants to deliver a project so I
thought I'd toss this email out in case any of this isn't addressed
in the newer libpq.

thanks,
-Alfred Perlstein - [bright(at)rush(dot)net|alfred(at)freebsd(dot)org]
Wintelcom systems administrator and programmer
- http://www.wintelcom.net/ [bright(at)wintelcom(dot)net]

Browse pgsql-interfaces by date

  From Date Subject
Next Message Joe Shevland 1999-12-11 06:47:36 Re: [INTERFACES] Large Object Exception/Serializing Java Objects
Previous Message gouri 1999-12-10 20:14:12 pgaccess