Sequential I/O Cost (was Re: A Better External Sort?)

On Wed, 2005-09-28 at 12:03 -0400, Ron Peacetree wrote:
> >From: "Jeffrey W. Baker" <jwbaker(at)acm(dot)org>
> >Perhaps I believe this because you can now buy as much sequential I/O
> >as you want. Random I/O is the only real savings.
> >
> 1= No, you can not "buy as much sequential IO as you want". Even if
> with an infinite budget, there are physical and engineering limits. Long
> before you reach those limits, you will pay exponentially increasing costs
> for linearly increasing performance gains. So even if you _can_ buy a
> certain level of sequential IO, it may not be the most efficient way to
> spend money.

This is just false. You can buy sequential I/O for linear money up to
and beyond your platform's main memory bandwidth. Even 1GB/sec will
severely tax memory bandwidth of mainstream platforms, and you can
achieve this rate for a modest cost.

I have one array that can supply this rate and it has only 15 disks. It
would fit on my desk. I think your dire talk about the limits of
science and engineering may be a tad overblown.