Re: [COMMITTERS] pgsql-server/ /configure /configure.in rc/incl ...

> > Absolutely! Is this a simple benchmark? Yup. Do I think it
> > simulates PostgreSQL? Eh, not particularly.

I think quite a few of these Q's would have been answered by reading
the code/Makefile....

> This would be on what OS?

FreeBSD, but it shouldn't matter. Any reasonably written VM should
have similar numbers (though BSD is generally regarded as having the
best VM, which, I think Linux poached not that long ago, iirc
::grimace::).

> What hardware?

My ultra-pathetic laptop with some fine - overly-noisy and can hardly
buildworld - IDE drives.

> What size test file?

In this case, only 72K. I've just updated the test program to use an
array of files though.

> Do the "iterations" mean so many reads of the entire file, or so
> many buffer-sized read requests?

In some cases, yes. With the file mmap()'ed, sorta. One of the test
cases (the one that did it in ~8s), mmap()'ed and munmap()'ed the file
every iteration and was twice as fast as the vanilla read() call.

> Did the mmap case actually *read* anything, or just map and unmap
> the file?

Nope, read it and wrote it out to stdout (which was redirected to
/dev/null).

> Also, what did you do to normalize for the effects of the test file
> being already in kernel disk cache after the first test?

That honestly doesn't matter too much since I wasn't testing the rate
of reading in files from my hard drive, only the OS's ability to
read/write pages of data around. In any case, I've updated my test
case to iterate through an array of files instead of just reading in a
copy of /etc/services. My laptop is generally a poor benchmark for
disk read performance given it takes 8hrs to buildworld, over 12hrs to
build mozilla, 18 for KDE, and about 48hrs for Open Office. :)
Someone with faster disks may want to try this and report back, but it
doesn't matter much in terms of relevancy for considering the benefits
of mmap(). The point is that there are calls that can be used that
substantially speed up read()'s and write()'s by allowing the VM to
align pages of data and give hints about its usage. For the sake of
argument re: the previously done tests, I'll reverse the order in
which I ran them and I bet dime to dollar that the times will be
identical.

% make ~/open_source/mmap_test
cp -f /etc/services ./services
gcc -O3 -finline-functions -fkeep-inline-functions -funroll-loops -DDEFAULT_READSIZE=1 -DDO_MMAP=1 -DDO_MMAP_ONCE=1 -o mmap-test mmap-test.c
/usr/bin/time ./mmap-test > /dev/null
Beginning tests with file: services

Page size: 4096
File read size is the same as the file size
Number of iterations: 100000
Start time: 1047064672.276544
Time: 1.281477

Completed tests
1.29 real 0.10 user 0.92 sys
gcc -O3 -finline-functions -fkeep-inline-functions -funroll-loops -DDEFAULT_READSIZE=1 -DDO_MMAP=1 -o mmap-test mmap-test.c
/usr/bin/time ./mmap-test > /dev/null
Beginning tests with file: services

Page size: 4096
File read size is the same as the file size
Number of iterations: 100000
Start time: 1047064674.266191
Time: 7.486622

Completed tests
7.49 real 0.41 user 6.01 sys
gcc -O3 -finline-functions -fkeep-inline-functions -funroll-loops -DDEFAULT_READSIZE=1 -o mmap-test mmap-test.c
/usr/bin/time ./mmap-test > /dev/null
Beginning tests with file: services

Page size: 4096
File read size is default read size: 65536
Number of iterations: 100000
Start time: 1047064682.288637
Time: 19.35214

Completed tests
19.04 real 0.88 user 15.43 sys
gcc -O3 -finline-functions -fkeep-inline-functions -funroll-loops -o mmap-test mmap-test.c
/usr/bin/time ./mmap-test > /dev/null
Beginning tests with file: services

Page size: 4096
File read size is the same as the file size
Number of iterations: 100000
Start time: 1047064701.867031
Time: 82.4294540875

Completed tests
81.57 real 2.10 user 69.55 sys

Here's the updated test that iterates through. Ooh! One better, the
files I've used are actual data files from ~pgsql. The new benchmark
iterates through the list of files and and calls bench() once for each
file and restarts at the first file after reaching the end of its
list (ARGV).

Whoa, if these tests are even close to real world, then we at the very
least should be mmap()'ing the file every time we read it (assuming
we're reading more than just a handful of bytes):

find /usr/local/pgsql/data -type f | /usr/bin/xargs /usr/bin/time ./mmap-test > /dev/null
Page size: 4096
File read size is the same as the file size
Number of iterations: 100000
Start time: 1047071143.463360
Time: 12.109530

Completed tests
12.11 real 0.36 user 6.80 sys

find /usr/local/pgsql/data -type f | /usr/bin/xargs /usr/bin/time ./mmap-test > /dev/null
Page size: 4096
File read size is default read size: 65536
Number of iterations: 100000
.... [been waiting here for >40min now....]

Ah well, if these tests finish this century, I'll post the results in
a bit, but it's pretty clearly a win. In terms of the data that I'm
copying, I'm copying ~700MB of data from my test DB on my laptop. I
only have 256MB of RAM so I can pretty much promise you that the data
isn't in my system buffers. If anyone else would like to run the
tests or look at the results, please check it out:

o1 and o2 should be the only targets used if FILES is bigger than the
RAM on the system. o3's by far and away the fastest, but only in rare
cases will a DBA have more RAM than data. But, as mentioned earlier,
the LRU cache could easily be modified to munmap() infrequently
accessed files to keep the size of mmap()'ed data down to a reasonable
level.

The updated test programs are at:

http://people.FreeBSD.org/~seanc/mmap_test/

-sc

--
Sean Chittenden