Re: amazon ec2

On Tue, May 3, 2011 at 2:09 PM, Alan Hodgson <ahodgson(at)simkin(dot)ca> wrote:

> On May 3, 2011 12:43:13 pm you wrote:
> > On May 3, 2011, at 8:41 PM, Alan Hodgson wrote:
> > > I am also interested in tips for this. EBS seems to suck pretty bad.
> >
> > Alan, can you elaborate? Are you using PG on top of EBS?
> >
>
> Trying to, yes.
>
> Let's see ...
>
> EBS volumes seem to vary in speed. Some are relatively fast. Some are
> really
> slow. Some fast ones become slow randomly. Some are fast attached to one
> instance, but really slow attached to another.
>
>
I ran pgbench tests late last year comparing EC2, GoGrid, a 5 year-old lab
server and a new server. Whether I used a stock postgresql.conf or tweaked,
the current 8.4 or 9.0, or varied the EC2 instance size EC2 was always at
the bottom ranging from 409.834 to 693.100 tps. GoGrid's pgbench TPS
numbers in similar tests were, on average, 3X that of EC2 (1,399.550 to
1,631.887 tps). The tests I conducted were small with 10 connections and
total 5,000 transactions. The single variable that helped pgbench tests in
EC2 was to select an instance size where the number of cores was equal to or
greater than the number of connections I used in the tests however this only
improved things slightly (715.931 tps).

For comparisons purposes, I ran the same tests on a 24-way X5650 with 12 GB
and SAS RAID 10. This server typically ranged from 2,188.348 to 2,216.377
tps.

I attributed GoGrids superior performance over EC2 as EC2 simply
being over-allocated but that's just speculation on my part. To test my
theory, I had wanted to put the database on a ramdisk, or like device, in
EC2 and GoGrid but never got around to it.

> Fast being a relative term, though. The fast ones seem to be able to do
> maybe
> 400 random IOPS. And of course you can only get about 80MB/sec sequential
> access to them on a good day.
>
> Which is why I'm interested in how other people are doing it. So far EC2
> doesn't seem well suited to running databases at all.
>
>
I was doing this perhaps to convince management to give me some time to
validate our software (PG backed) on some of the cloud providers but with
those abysmal numbers I didn't even bother at the time. I may revisit at
some point b/c I know Amazon at least has been making architecture
adjustments and updates.

Greg