Re: Amazon EC2 CPU Utilization

Mike Bresnahan wrote:
> I have deployed PostgresSQL 8.4.1 on a Fedora 9 c1.xlarge (8x1 cores) instance
> in the Amazon E2 Cloud. When I run pgbench in read-only mode (-S) on a small
> database, I am unable to peg the CPUs no matter how many clients I throw at it.
> In fact, the CPU utilization never drops below 60% idle. I also tried this on
> Fedora 12 (kernel 2.6.31) and got the same basic result. What's going on here?
> Am I really only utilizing 40% of the CPUs? Is this to be expected on virtual
> (xen) instances?
> tps = 19663.841772 (including connections establishing

Looks to me like you're running into a general memory bandwidth issue
here, possibly one that's made a bit worse by how pgbench works. It's a
somewhat funky workload Linux systems aren't always happy with, although
one of your tests had the right configuration to sidestep the worst of
the problems there. I don't see any evidence that pgbench itself is a
likely suspect for the issue, but it does shuffle a lot of things around
in memory relative to transaction time when running this small
select-only test, and clients can get stuck waiting for it when that
happens.

To put your results in perspective, I would expect to get around 25K TPS
running the pgbench setup/test you're doing on a recent 4-core/single
processor system, and around 50K TPS is normal for an 8-core server
doing this type of test. And those numbers are extremely sensitive to
the speed of the underlying RAM even with the CPU staying the same.

I would characterize your results as "getting about 1/2 of the
CPU+memory performance of an install on a dedicated 8-core system".
That's not horrible, as long as you have reasonable expectations here,
which is really the case for any virtualized install I think. I'd
actually like to launch a more thorough investigation into this
particular area, exactly how the PostgreSQL bottlenecks shift around on
EC2 compared to similar dedicated hardware, if I found a sponsor for it
one day. A bit too much work to do it right just for fun.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.com