Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins

On 11/17/10 15:26, Eric Comeau wrote:
> This is not directly a PostgreSQL performance question but I'm hoping
> some of the chaps that build high IO PostgreSQL servers on here can help.
>
> We build file transfer acceleration s/w (and use PostgreSQL as our
> database) but we need to build a test server that can handle a sustained
> write throughput of 1,25 GB for 5 mins.

Just to clarify: you need 1.25 GB/s write throughput?

For one thing, you need not only fast storage but also a fast CPU and
file system. If you are going to stream this data directly over the
network in a single FTP-like session, you need fast single-core
performance (so buy the fastest low-core-count CPU possible) and a file
system which doesn't interfere much with raw data streaming. If you're
using Linux I'd guess either something very simple like ext2 or complex
but designed for the task like XFS might be best. On FreeBSD, ZFS is
great for streaming but you'll spend a lot of time tuning it :)

From the hardware POW, since you don't really need high IOPS rates, you
can go much cheaper with a large number of cheap desktop drives than
with SSD-s, if you can build something like this:
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/

You don't need the storage space here, but you *do* need many drives to
achieve speed in RAID (remember to overdesign and assume 50 MB/s per drive).