Re: [PATCHES] COPY FROM performance improvements

On Fri, 22 Jul 2005 12:28:43 -0700
"Luke Lonergan" <llonergan(at)greenplum(dot)com> wrote:

> Joshua,
>
> On 7/22/05 10:11 AM, "Joshua D. Drake" <jd(at)commandprompt(dot)com> wrote:
> > The database server is a PE (Power Edge) 6600
> >
> > Database Server IO:
> >
> > [root(at)master root]# /sbin/hdparm -tT /dev/sda
> >
> > /dev/sda:
> > Timing buffer-cache reads: 1888 MB in 2.00 seconds = 944.00 MB/sec
> > Timing buffered disk reads: 32 MB in 3.06 seconds = 10.46 MB/sec
> >
> > Second Database Server IO:
> >
> > [root(at)pq-slave root]# /sbin/hdparm -tT /dev/sda
> >
> > /dev/sda:
> > Timing buffer-cache reads: 1816 MB in 2.00 seconds = 908.00 MB/sec
> > Timing buffered disk reads: 26 MB in 3.11 seconds = 8.36 MB/sec
> > [root(at)pq-slave root]#
>
> Can you post the "time dd if=/dev/zero of=bigfile bs=8k count=500000"
> results? Also do the reverse (read the file) with "time dd if=bigfile
> of=/dev/null bs=8k".
>
> I think you are observing what we've known for a while, hardware RAID is
> horribly slow. We've not found a hardware RAID adapter of this class yet
> that shows reasonable read or write performance. The Adaptec 2400R or the
> LSI or others have terrible internal I/O compared to raw SCSI with software
> RAID, and even the CPU usage is higher on these cards while doing slower I/O
> than linux SW RAID.
>
> Notably - we've found that the 3Ware RAID controller does a better job than
> the low end SCSI RAID at HW RAID support, and also exports JBOD at high
> speeds. If you export JBOD on the low end SCSI RAID adapters, the
> performance is also very poor, though generally faster than using HW RAID.

Are there any recommendations for Qlogic controllers on Linux, scsi or
fiber channel? I might be able to my hands on some. I have pci-x slots
for AMD, Itanium, or POWER5 if the architecture makes a difference.

Mark