Re: Huge Data sets, simple queries

On Fri, Jan 27, 2006 at 07:05:04PM -0800, Luke Lonergan wrote:
> Sounds like you are running into the limits of your disk subsystem. You are
> scanning all of the data in the transactions table, so you will be limited
> by the disk bandwidth you have ? and using RAID-10, you should divide the
> number of disk drives by 2 and multiply by their indiividual bandwidth
> (around 60MB/s) and that?s what you can expect in terms of performance. So,
> if you have 8 drives, you should expect to get 4 x 60 MB/s = 240 MB/s in
> bandwidth. That means that if you are dealing with 24,000 MB of data in the
> ?transactions? table, then you will scan it in 100 seconds.

Why divide by 2? A good raid controller should be able to send read
requests to both drives out of the mirrored set to fully utilize the
bandwidth. Of course, that probably won't come into play unless the OS
decides that it's going to read-ahead fairly large chunks of the table
at a time...

Also, some vmstat output would certainly help clarify where the
bottleneck is...
--
Jim C. Nasby, Sr. Engineering Consultant jnasby(at)pervasive(dot)com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461