Re: Huge Data sets, simple queries

PFC,

On 2/1/06 1:01 AM, "PFC" <lists(at)peufeu(dot)com> wrote:

> 3- Read one block of A, then one block of B, then one block of A, etc.
> Essentially this is the same as the threaded case, except there's only one
> thread.
> 53 seconds total (with heavy seeking noise from the hdd).
>
> I was half expecting 3 to take the same as 2. It simulates, for
> instance,
> scanning a table and its index, or scanning 2 sort bins. Well, maybe one
> day...

This is actually interesting overall - I think what this might be showing is
that the Linux SW RAID1 is alternating I/Os to the mirror disks from
different processes (LWP or HWP both maybe?), but not within one process.

> It would be nice if the Kernel had an API for applications to tell it
> "I'm gonna need these blocks in the next seconds, can you read them in the
> order you like (fastest), from whatever disk you like, and cache them for
> me please; so that I can read them in the order I like, but very fast ?"

More control is always good IMO, but for now there's I/O reordering in the
SCSI layer and readahead tuning. There is POSIX fadvise() also to tell the
underlying I/O layer what the access pattern looks like.

- Luke