Skip site navigation (1) Skip section navigation (2)

Re: pgbench could not send data to client: Broken pipe

From: David Kerr <dmk(at)mr-paradox(dot)net>
To: Greg Smith <greg(at)2ndquadrant(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-performance(at)postgresql(dot)org
Subject: Re: pgbench could not send data to client: Broken pipe
Date: 2010-09-08 19:56:54
Message-ID: 20100908195654.GB49941@mr-paradox.net (view raw or flat)
Thread:
Lists: pgsql-performance
On Wed, Sep 08, 2010 at 03:27:34PM -0400, Greg Smith wrote:
- Tom Lane wrote:
- >As of the 9.0 release, it's possible to run pgbench in a "multi thread"
- >mode, and if you forced the subprocess rather than thread model it looks
- >like the select() limit would be per subprocess rather than global.
- >So I think you could get above the FD_SETSIZE limit with a bit of
- >hacking if you were using 9.0's pgbench.  No chance with 8.3 though.
- >  
- 
- I believe David can do this easily enough by compiling a 9.0 source code 
- tree with the "--disable-thread-safety" option.  That's the simplest way 
- to force the pgbench client to build itself using the multi-process 
- model, rather than the multi-threaded one.
- 
- It's kind of futile to run pgbench simulating much more than a hundred 
- or two clients before 9.0 anyway.  Without multiple workers, you're 
- likely to just run into the process switching limitations within pgbench 
- itself rather than testing server performance usefully.  I've watched 
- the older pgbench program fail to come close to saturating an 8 core 
- server without running into its own limitations first.
- 
- You might run a 9.0 pgbench client against an 8.3 server though, if you 
- did the whole thing starting from pgbench database initialization over 
- again--the built-in tables like "accounts" changed to "pgbench_accounts" 
- in 8.4.  That might work, can't recall any changes that would prevent 
- it; but as I haven't tested it yet I can't say for sure.

Thanks, I compiled the 9.0 RC1 branch with the --disable-thread-safety option
and ran PG bench on my 8.3 DB it seemed to work fine, 

However, MAXCLIENTS is still 1024, if i hack it to switch it up to 2048 i 
get this:
starting vacuum...end.
select failed: Bad file descriptor  <---------------
transaction type: TPC-B (sort of)
scaling factor: 1
query mode: simple
number of clients: 1900
number of threads: 1
number of transactions per client: 10
number of transactions actually processed: 3723/19000
tps = 52.007642 (including connections establishing)
tps = 82.579077 (excluding connections establishing)


I'm not sure what Tom is referring to with the select(2) limitation, maybe I'm running
into it (where do i find that? /usr/include/sys/select.h? )

should i be running pgbench differently? I tried increasing the # of threads
but that didn't increase the number of backend's and i'm trying to simulate
2000 physical backend processes.

thanks

Dave

In response to

Responses

pgsql-performance by date

Next:From: David KerrDate: 2010-09-08 19:58:17
Subject: Re: pgbench could not send data to client: Broken pipe
Previous:From: Tom LaneDate: 2010-09-08 19:44:36
Subject: Re: pgbench could not send data to client: Broken pipe

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group