Re: amazon ec2

Mark Rostron wrote:
> the success/failure of it depends on your typical query activity, the
> size of your critical result set, and whether you are able to get
> enough RAM to make this work.

Basically, it all comes down to "does the working set of data I access
frequently fit in RAM?" If it does, it's possible to get reasonable
performance out of an EC2 instance. The EBS disks are so slow, both on
average and particularly in cases where you have contention with other
users slowing you down, that any situation where you have to use them is
never going to work well. If most of the data fits in RAM, and the CPU
resources available to your instance are sufficient to service your
queries, you might see acceptable performance.

> greenplum is also a better parallel machine than postgres, so
> combining the cache technique above with greenplum compression and
> parallel query, i have been able to get 20:1 reduction in response
> times for some of our queries.

I've also seen over a 20:1 speedup over PostgreSQL by using Greenplum's
free Community Edition server, in situations where its column store +
compression features work well on the data set. That's easiest with an
append-only workload, and the data set needs to fit within the
constraints where indexes on compressed data are useful. But if you fit
the use profile it's good at, you end up with considerable ability to
trade-off using more CPU resources to speed up queries. It effectively
increases the amount of data that can be cached in RAM by a large
multiple, and in the EC2 context (where any access to disk is very slow)
it can be quite valuable. My colleague Gabrielle wrote something about
setting this up on an earlier version of Greenplum's software at
http://blog.2ndquadrant.com/en/2010/03/installing-greenplum-sne-ec2.html
that gives an idea how that was setup.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books