error encountered in windows when libpq sends large query string to server

From: Ranen Ghosh <rghosh(at)westernavionics(dot)com>
To: pgsql-interfaces(at)postgresql(dot)org
Subject: error encountered in windows when libpq sends large query string to server
Date: 2005-11-03 15:22:55
Message-ID: 436A2B4F.6000005@westernavionics.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-interfaces

A winsock error is encountered when attempting to send a large query
string (over 60 megabytes after escape characters were added) to a
postgresql server from windows XP. The error was observed in using the
pyPgSQL python binding to libpq, but was traced to libpq.

In python, the error was:

libpq.OperationalError?
<https://www.comcube.ca/trac/comcube/wiki/OperationalError> could not
send data to server: No buffer space available (0x00002747/10055)

<traceback object at 0x00CC5260>

from postgresql-8.0.3/src/interfaces/libpq/win32.c: WSAENOBUFS, "No
buffer space available"
from
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/windows_sockets_error_codes_2.asp:
WSAENOBUFS 10055

The problem is client side: when a smaller query string is successfully
sent, a series of TCP packets are exchanged between the client and the
server; but when the large query string is sent, there are no TCP
packets exchanged. The query-string-size at which the error occurs is
somewhere around 64 megabytes after escaping.

The problem occurs in postgresql-8.0.3/src/interfaces/libpq/fe-misc.c,
in pqSendSome(), which calls pqsecure_write() in
postgresql-8.0.3/src/interfaces/libpq/fe-secure.c. In this function, the
send() function call fails when the length of the buffer reaches
approximately 64M.

The code for postgresql-8.1beta3 does not seem to implement the above
mentioned functions differently.

As the error is client side, and the error message is a winsock error, I
had thought that the most likely solution was changing an attribute of
the socket (perhaps with setsockopt()) to expand the buffer space
available). The default value of SO_SNDBUF is 8192. Changing this value
to 0 or to 16384 (and confirming that the value had in fact changed
internally) had no effect on the problem, however.

Browse pgsql-interfaces by date

  From Date Subject
Next Message Vita Voom Software 2005-11-03 16:42:48 Released pgExpress Driver v3.50
Previous Message Basile STARYNKEVITCH 2005-11-02 22:34:49 libpq, getting array from table... (PostGreSQL8)