Benchmark with FreeBSD 8.0 and pgbench

This is not a rigorous test and should not be used as a direct comparison
between
operating systems. My objective was to estimate the ZFS toll in Postgres,
and
what to expect in performance, comparing to the old server this machine will
be replacing. Not all possible configurations were tested, and the CentOS
benchmark was made out of curiosity.

All tests were made using the default instalation, without any additional
tuning,
except those mentioned in custom configuration. The installation media used
was the
8.0-STABLE-201002 DVD (amd64). The server was rebooted between configuration
changes, but NOT between successive runs.
There are some inconsistencies between results,probably due to caching.
Disabling atime surprisingly results in slower runs, in both ZFS and UFS.
The CentOS test was made with the same default configuration file used in
FreeBSD.

Any additional hints are welcome, but I probably won't be able to re-run
the tests :)

Regards,
João Pinheiro

Machine Specs:
- Hp Proliant ML330
- 1x Intel(R) Xeon CPU E5504 2.0Ghz
- 12GB RAM
- 4 x 250GB SATA HD 7.2 (RAID 10)
- P410 Raid controller w/512MB cache/battery option

OS and Postgres Custom Configuration (used on tests marked as custom):

postgresql.conf:
----------------------
max_connections = 500
work_mem = 16MB
shared_buffers = 3072MB

/etc/sysctl.conf
---------------------
kern.ipc.shmmax=3758096384
kern.ipc.shmall=917504
kern.ipc.shm_use_phys=1

/boot/loader.conf
-----------------------
kern.ipc.semmni=512
kern.ipc.semmns=1024
kern.ipc.semmnu=512

FreeBSD Setup:
$ uname -a
FreeBSD beastie 8.0-STABLE-201002 FreeBSD 8.0-STABLE-201002 #0: Tue Feb 16
21:05:59 UTC 2010
root(at)mason(dot)cse(dot)buffalo(dot)edu:/usr/obj/usr/src/sys/GENERIC amd64

CentOS setup:
[root(at)dhcppc0 test]# uname -a
Linux dhcppc0 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64
x86_64 x86_64 GNU/Linux
iptables: off
selinux: off

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS, postgresql84-server from package, default config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2363.017640 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2310.928229 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2042.681002 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1891.323153 (including connections
establishing)

bonnie++:

Version 1.96 ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
beastie 24G 700 99 170476 22 59826 11 967 78 156937 15
475.7 5
Latency 12177us 500ms 830ms 291ms 145ms
269ms
Version 1.96 ------Sequential Create------ --------Random
Create--------
beastie -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
20 35329 48 +++++ +++ +++++ +++ 19828 86 +++++ +++ +++++
+++
Latency 26925us 15us 83us 547ms 80us
60us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS,noatime, postgresql84-server from package, default
conf
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2287.987630 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2255.514875 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2098.280816 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1871.193058 (including connections
establishing)

bonnie++:

Version 1.96 ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
beastie 24G 679 99 170965 22 58682 11 942 76 157156 15
498.7 6
Latency 12881us 569ms 1171ms 681ms 155ms
242ms
Version 1.96 ------Sequential Create------ --------Random
Create--------
beastie -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
20 36118 49 +++++ +++ 27231 99 +++++ +++ +++++ +++ +++++
+++
Latency 32705us 81us 537ms 14280us 25us
82us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS, postgresql84-server from package, custom config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2320.509732 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2289.853556 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2089.112777 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1921.034254 (including connections
establishing)
pgbench -c 50 -t 1000 testdb : 1574.231270 (including connections
establishing)
pgbench -c 100 -t 1000 testdb : 1096.761450 (including connections
establishing)
pgbench -c 200 -t 1000 testdb : 256.443268 (including connections
establishing)
pgbench -c 300 -t 1000 testdb : 69.174219 (including connections
establishing)

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS, postgresql84-server from ports, default config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 1676.760766 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2295.344502 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2066.721058 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1887.196064 (including connections
establishing)

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, UFS, postgresql84-server from ports, custom config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2642.962465 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2253.349999 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2037.050831 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1918.147928 (including connections
establishing)
pgbench -c 50 -t 1000 testdb : 1773.778845 (including connections
establishing)
pgbench -c 100 -t 1000 testdb : 1153.303103 (including connections
establishing)
pgbench -c 200 -t 1000 testdb : 517.648628 (including connections
establishing)
pgbench -c 300 -t 1000 testdb : 97.442573 (including connections
establishing)

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE, ZFS, default block size
-------------------------------------------------------------------------------

bonnie++:

Version 1.96 ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
beastie 24G 120 99 154421 31 81917 16 327 98 196125 16
175.7 4
Latency 222ms 13439ms 12749ms 163ms 1034ms
655ms
Version 1.96 ------Sequential Create------ --------Random
Create--------
beastie -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
20 25979 99 32036 55 16342 99 24022 99 +++++ +++ 21384
99
Latency 13639us 107ms 977us 31274us 144us
257us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS (8k block),postgresql84-server from package,default
conf.
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2380.858280 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2135.241727 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2002.773173 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1692.844103 (including connections
establishing)

bonnie++:

Version 1.96 ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
beastie 24G 107 99 119235 22 50495 17 321 98 143257 25
201.5 4
Latency 82593us 13575ms 11274ms 74329us 1398ms
666ms
Version 1.96 ------Sequential Create------ --------Random
Create--------
beastie -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
20 24960 99 +++++ +++ 21449 99 22501 99 +++++ +++ 22156
99
Latency 17467us 137us 167us 37528us 110us
147us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS-atime=off,postgresql84-server from package,default
conf.
-------------------------------------------------------------------------------

bonnie++:

Version 1.96 ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
beastie 24G 114 99 151906 32 81971 17 295 90 192878 16
173.5 5
Latency 286ms 15766ms 13315ms 1062ms 433ms
727ms
Version 1.96 ------Sequential Create------ --------Random
Create--------
beastie -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
20 25295 98 15774 78 12823 99 20979 99 +++++ +++ 22466
99
Latency 30857us 98288us 1321us 35519us 113us
144us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS-8k/atime=off,postgresql84-server from package,default
conf.
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2235.009456 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 1915.160680 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 1824.546833 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1663.443537 (including connections
establishing)

bonnie++:

Version 1.96 ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
beastie 24G 109 99 123147 24 51160 17 327 97 117123 21
191.4 4
Latency 153ms 17900ms 9873ms 164ms 669ms
600ms
Version 1.96 ------Sequential Create------ --------Random
Create--------
beastie -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
20 25324 99 +++++ +++ 18689 99 20837 99 18828 100 5055
100
Latency 13866us 48457us 241us 30971us 199us
596us

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS-8k,postgresql84-server from package,custom config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2438.179353 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 1949.016648 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 1570.176692 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1683.720510 (including connections
establishing)
pgbench -c 50 -t 1000 testdb : 1481.249222 (including connections
establishing)
pgbench -c 100 -t 1000 testdb : 1034.946222 (including connections
establishing)
pgbench -c 200 -t 1000 testdb : 288.125818 (including connections
establishing)
pgbench -c 300 -t 1000 testdb : 57.924377 (including connections
establishing)

-------------------------------------------------------------------------------
FreeBSD 8.0-STABLE,ZFS-8k,postgresql84-server from ports,custom config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2252.105155 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2065.147771 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 1762.356143 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 1832.577548 (including connections
establishing)
pgbench -c 50 -t 1000 testdb : 1571.061549 (including connections
establishing)
pgbench -c 100 -t 1000 testdb : 1091.865609 (including connections
establishing)
pgbench -c 200 -t 1000 testdb : 476.009429 (including connections
establishing)
pgbench -c 300 -t 1000 testdb : 101.918069 (including connections
establishing)

-------------------------------------------------------------------------------
CentOS 5.4, EXT3,postgresql84-server from package, default config
-------------------------------------------------------------------------------

pgbench -c 10 -t 1000 testdb : 2407.934433 (including connections
establishing)
pgbench -c 20 -t 1000 testdb : 2369.103760 (including connections
establishing)
pgbench -c 30 -t 1000 testdb : 2165.642174 (including connections
establishing)
pgbench -c 40 -t 1000 testdb : 2060.112114 (including connections
establishing)

bonnie++:

Version 1.94 ------Sequential Output------ --Sequential Input-
--Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
dhcppc0 24G 548 99 167274 30 73785 11 1088 73 217100 11
612.3 17
Latency 15995us 957ms 575ms 576ms 111ms
73868us
Version 1.94 ------Sequential Create------ --------Random
Create--------
dhcppc0 -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
20 39388 94 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++
+++
Latency 114us 725us 750us 182us 78us
370us

--
View this message in context: http://old.nabble.com/Benchmark-with-FreeBSD-8.0-and-pgbench-tp28569544p28569544.html
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.