Re: Load experimentation

Ben Brehmer wrote:
> Since my input data is split into about 200 files (3GB each), I could
> potentially spawn one load command for each file. What would be the
> maximum number of input connections Postgres can handle without
> bogging down?
You can expect to easily get one loader process per real CPU going.
Beyond that, it depends on how CPU intensive they all are and what the
resulting I/O rate out of the combination is. You're probably going to
run out of CPU on a loading job long before you hit any of the other
limits in this area, and potentially you could run out of disk
throughput on a cloud system before that. PostgreSQL isn't going to bog
down on a connection basis until you've reached several hundred of them,
your loader will be lucky to hit 10 active processes before it grinds to
a halt on some physical resources unrelated to general database scaling.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.com