Libpq optimization

From: "Alon Goldshuv" <agoldshuv(at)greenplum(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Libpq optimization
Date: 2005-10-26 17:20:41
Message-ID: BF853329.A5F5%agoldshuv@greenplum.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

In the libpq COPY interface function PQputCopyData():

/*
* Check for NOTICE messages coming back from the server. Since the
* server might generate multiple notices during the COPY, we have to
* consume those in a reasonably prompt fashion to prevent the comm
* buffers from filling up and possibly blocking the server.
*/
if (!PQconsumeInput(conn))
return -1; /* I/O failure */
parseInput(conn);

I moved it to a different location, just a bit further, after the check for
"is output buffer full and we are ready to flush?" in the same function

if ((conn->outBufSize - conn->outCount - 5) < nbytes)
{
<here>
....
....
}

As the code comment suggests, it is extremely important to consume incoming
messages from the server to prevent deadlock. However we should only worry
about it before sending data out. Most calls to PQputCopyData don't actually
send any data but just place it in the out buffer and return. Therefore we
can perform the consumeinput/parseinput right before flushing, instead of
reading from the server every time we call PQputCopyData and not send
anything (which happens probably in 99% of the time).

Right? Or am I missing something.

This change improves COPY performance.

thx
Alon.

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kevin Grittner 2005-10-26 17:30:58 Re: [BUGS] BUG #1993: Adding/subtracting negative
Previous Message strk 2005-10-26 17:19:10 IMMUTABLE bug ?