Skip site navigation (1) Skip section navigation (2)

Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins

From: Dimitri <dimitrik(dot)fr(at)gmail(dot)com>
To: Greg Smith <greg(at)2ndquadrant(dot)com>
Cc: Eric Comeau <Eric(dot)Comeau(at)signiant(dot)com>, Merlin Moncure <mmoncure(at)gmail(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins
Date: 2010-11-20 10:16:42
Message-ID: AANLkTimC_VVWvg46MZBdXPtMPms0uHiUmLNPkNOhDR+F@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-performance
You may also try the Sun's F5100 (flash storage array) - you may
easily get 700 MB/s just with a single I/O stream (single process), so
just with 2 streams you'll get your throughput.. - The array has 2TB
total space and max throughput should be around 4GB/s..

Rgds,
-Dimitri


On 11/18/10, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> Eric Comeau wrote:
>> Ideally 1 large file, but it may have to be multiple. We find that if
>> we send multiple files it just causes the disk to thrash more so we
>> get better throughput by sending one large file.
>
> If it's really one disk, sure.  The problem you're facing is that your
> typical drive controller is going to top out at somewhere between 300 -
> 500MB/s of sequential writes before it becomes the bottleneck.  Above
> somewhere between 6 and 10 drives attached to one controller on current
> hardware, adding more to a RAID-0 volume only increases the ability to
> handle seeks quickly.  If you want to try and do this with traditional
> hard drives, I'd guess you'd need 3 controllers with at least 4
> short-stroked drives attached to each to have any hope of hitting
> 1.25GB/s.  Once you do that, you'll run into CPU time as the next
> bottleneck.  At that point, you'll probably need one CPU per controller,
> all writing out at once, to keep up with your target.
>
> The only popular hardware design that comes to mind aimed at this sort
> of thing was Sun's "Thumper" design, most recently seen in the Sun Fire
> X4540.  That put 8 controllers with 6 disks attached to each, claiming
> "demonstrated up to 2 GB/sec from disk to network".  It will take a
> design like that, running across multiple controllers, to get what
> you're looking for on the disk side--presuming everything else keeps up.
>
> One of the big SSD-on-PCI-e designs mentioned here already may very well
> end up being a better choice for you here though, as those aren't going
> to require quite as much hardware all get wired up.
>
> --
> Greg Smith   2ndQuadrant US    greg(at)2ndQuadrant(dot)com   Baltimore, MD
> PostgreSQL Training, Services and Support        www.2ndQuadrant.us
> "PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books
>
>

In response to

pgsql-performance by date

Next:From: Bob LunneyDate: 2010-11-20 16:27:51
Subject: Re: best db schema for time series data?
Previous:From: tvDate: 2010-11-20 04:43:18
Subject: Re: autovacuum blocks the operations of other manual vacuum

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group