From: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
---|---|
To: | Martijn van Oosterhout <kleptog(at)svana(dot)org> |
Cc: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: Producer/Consumer Issues in the COPY across network |
Date: | 2008-02-28 01:57:49 |
Message-ID: | 1204163869.4252.758.camel@ebony.site |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Tue, 2008-02-26 at 12:29 +0100, Martijn van Oosterhout wrote:
> > When we're running a COPY over a high latency link then network time is
> > going to become dominant, so potentially, running COPY asynchronously
> > might help performance for loads or initial Slony configuration. This is
> > potentially more important on Slony where we do both a PQgetCopyData()
> > and PQputCopyData() in a tight loop.
>
> When you check the packets being sent, are you showing only one record
> being sent per packet? If so, there's your problem.
I've not inspected the packet flow. It seemed easier to ask.
> > I also note that PQgetCopyData always returns just one row. Is there an
> > underlying buffering between the protocol (which always sends one
> > message per row) and libpq (which is one call per row)? It seems
> > possible for us to request a number of rows from the server up to a
> > preferred total transfer size.
>
> AIUI the server merely streams the rows to you, the client doesn't get
> to say how many :)
Right, but presumably we generate a new message per PQgetCopyData()
request? So my presumption is we need to wait for that to be generated
each time?
--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com
From | Date | Subject | |
---|---|---|---|
Next Message | Babu, Gabriel Suresh | 2008-02-28 07:02:28 | ES7000 Windows 2003 server 64bit processor |
Previous Message | Tom Lane | 2008-02-28 00:53:48 | Buildfarm member gypsy_moth seems not to like alignment patch |