Re: Amazon EC2 | Any recent developments

Hi,

> So, when a cloud machine fails does it get de-allocated/wiped out? or
> does it is it still out there in a bad state? how do you recover your
> data?

It depends. Sometimes it dies and you can't do anything with it. In
others you can restart it. As we store the data on EBS (which is a
network storage in AMazon's cloud), the data is not related to the
instance directly. And that's the beauty of it. If a machine dies I
can launch a replacement machine (it takes about 3-5 minutes for it to
boot and be ready), or in some cases I just have a replacement
instance waiting. I then mount the EBS volume holding the Postgres
data, let the server do the recovery if needed and I'm back online. No
need to replace a chip, go to the DC, etc...

In case the volume got corrupted (a very rare situation, as the EBS
volumes are very durable), there are snapshots I can recover from and
the WAL files I stream to another storage system (Amazon's S3). I have
some systems where I take daily tar backups of the PG directory and
ship those to a separate storage.

There is no doubt you can get a much stronger machine by building your
own hardware, but the ability to easily recover, and easily launch and
cluster are a huge advantage for the cloud. It does takes getting used
to, though. You need to think of machines as expendable, and plan for
easy failure preparation and replacement. It does make you really
prepare and test your recovery strategies. I know of too many
companies that just trust the DB to be ok. And it is most of the time,
but when a catastrophe happens, recovery is a long and risky process.

Bye,

Guy.

--
Family management on rails: http://www.famundo.com
My development related blog: http://devblog.famundo.com