Skip site navigation (1) Skip section navigation (2)

Re: TCP network cost

From: "Ross J(dot) Reedstrom" <reedstrm(at)rice(dot)edu>
To: pgsql-performance <pgsql-performance(at)postgresql(dot)org>
Subject: Re: TCP network cost
Date: 2009-02-23 19:43:25
Message-ID: 20090223194325.GB14802@cooker (view raw or flat)
Thread:
Lists: pgsql-performance
On Thu, Feb 19, 2009 at 02:09:04PM +0100, PFC wrote:
> 
> >python w/ psycopg (or psycopg2), which wraps libpq. Same results w/
> >either version.
> 
> 	I've seen psycopg2 saturate a 100 Mbps ethernet connection (direct  
> connection with crossover cable) between postgres server and client during  
> a benchmark... I had to change the benchmark to not retrieve a large TEXT  
> column to remove this bottleneck... this was last year so versions are  
> probably different, but I don't think this matters a lot...

Here's the core of the problem: I in fact need to transfer exactly that:
a large single field (bytea in my case). I suspect psycopg[12] is having
issues w/ memory allocation, but that's just an unsupported gut feeling.

The final upshot is that I need to restructure my config to use the
large-object API (and hence a snapshot of psycopg2) to get decent
throughput. 

> 	You should test with sending a large (>100 MB) amount of data 
> 	through  Netcat. This should give you your maximum wire speed. Use 
> /dev/null as the  test file, and use "pv" (pipe viewer) to measure 
> throughput :
> 
> box 1 : pv < /dev/zero | nc -lp 12345
> box 2 : nc (ip) 12345 >/dev/null
> 
> 	On gigabit lan you should get 100 MB/s, on 100BaseT about 10 MB/s. 

112 MB/s, and 233 MB/s for localhost. Thanks for the pointer to pv:
looks like a nice tool. Investigating this problem has lead me to a
number of nice 'old school' tools: the other is tcptrace and xplot.org.
I've been hand reading tcpdump output, or clicking around in
ethereal/wireshark. I like tcptrace's approach.

Ross
-- 
Ross Reedstrom, Ph.D.                                 reedstrm(at)rice(dot)edu
Systems Engineer & Admin, Research Scientist        phone: 713-348-6166
The Connexions Project      http://cnx.org            fax: 713-348-3665
Rice University MS-375, Houston, TX 77005
GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E  F888 D3AE 810E 88F0 BEDE


In response to

pgsql-performance by date

Next:From: davidDate: 2009-02-23 21:02:38
Subject: Re: postgreSQL performance 8.2.6 vs 8.3.3
Previous:From: Ross J. ReedstromDate: 2009-02-23 19:42:12
Subject: Re: TCP network cost

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group