Re: Hardware/OS recommendations for large databases (

Greg,

On 11/18/05 11:07 AM, "Greg Stark" <gsstark(at)mit(dot)edu> wrote:

> That said, 130MB/s is nothing to sneeze at, that's maxing out two high end
> drives and quite respectable for a 3-disk stripe set, even reasonable for a
> 4-disk stripe set. If you're using 5 or more disks in RAID-0 or RAID 1+0 and
> only getting 130MB/s then it does seem likely the cpu is actually holding you
> back here.

With an FC array, it's undoubtedly more like 14 drives, in which case
130MB/s is laughable. On the other hand, I wouldn't be surprised if it were
a single 200MB/s Fibre Channel attachment.

It does make you wonder why people keep recommending 15K RPM drives, like it
would help *not*.

> Still it doesn't show Postgres being nearly so CPU wasteful as the original
> poster claimed.

It's partly about waste, and partly about lack of a concurrent I/O
mechanism. We've profiled it for the waste, we've implemented concurrent
I/O to prove the other point.

>> It's all in the kernel either way; using a different scheduler or file
>> system would change that result. Even better would be using direct IO to not
>> flush everything else from memory and avoid some memory copies from kernel
>> to user space. Note that almost none of the time is user time. Changing
>> postgresql won't change the cpu useage.
>
> Well changing to direct i/o would still be changing Postgres so that's
> unclear. And there are plenty of more mundane ways that Postgres is
> responsible for how efficiently or not the kernel is used. Just using fewer
> syscalls to do the same amount of reading would reduce cpu consumption.

Bingo.

- Luke