Re: [HACKERS] sort on huge table

Tatsuo Ishii wrote:
>
> >... I agree this'd be worth trying. There could be some subtle effect
> >somewhere in RAID5 that's tripping things up. It'd also be useful if
> >someone could try it on similar RAID hardware with a non-Linux kernel.
>
> I have compared current with 6.5 using 1000000 tuple-table (243MB) (I
> wanted to try 2GB+ table but 6.5 does not work in this case). The
> result was strange in that current is *faster* than 6.5!
>
> RAID5
> current 2:29
> 6.5.2 3:15
>
> non-RAID
> current 1:50
> 6.5.2 2:13
>
> Seems my previous testing was done in wrong way or the behavior of
> sorting might be different if the table size is changed?

Or the behaviour of RAID5 changes at some size.

I have set up an IBM Netfinity server with specs similar to yours, except
that it has 1G memory and 5x9GB disks. The RAID controller is IBM ServeRAID.

It seems that when I try to write over 60 MB sequentially, the write
performance drops from over 50MB/s to under 2MB/s.

Maybe such behaviour would suggest that building an index and traversing
that could be faster than full sort ?

The same tests on my single Celeron 450 produced ~10MB/s writes
whatever the size.

> Anyway, here is my test script.
> First, edit Makefile to set DB and number of tuples. Then run type
> make. That's all.

I'll try to run it tonight (in GMT+2 tz).

Can't run it earlyer as it is a production site and a highly
visible web-server.

If I have the time I'll even try my index theory.

--------
Hannu