Re: Open request for benchmarking input

Jeff, Qingqing,

On 11/26/05 10:57 AM, "Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu> wrote:

>
> "Jeff Frost" <jeff(at)frostconsultingllc(dot)com> wrote
>>
>> Did you folks see this article on Slashdot with a fellow requesting input
>> on what sort of benchmarks to run to get a good Postgresql vs Mysql
>> dataset? Perhaps this would be a good opportunity for us to get some good
>> benchmarking done.
>> "The hardware I have available is as follows:
>>
>> * 2x dual Opteron 8G ram, 2x144G 15Krpm SCSI
>> * 2x dual Opteron 8G ram, 2x72G 15Krpm SCSI
>> * 1x dual Opteron 16G ram, 2x36G 15Krpm SCSI 16x400G 7200rpm SATA
>>

I suggest specifying a set of basic system / HW benchmarks to baseline the
hardware before each benchmark is run. This has proven to be a major issue
with most performance tests. My pick for I/O is bonnie++.

Your equipment allows you the opportunity to benchmark all 5 machines
running together as a cluster - this is important to measure maturity of
solutions for high performance warehousing. Greenplum can provide you a
license for Bizgres MPP for this purpose.

> (2) The hardware configuration may not reflect all potentials of PostgreSQL.
> For example, so far, PostgreSQL does not pay much attention in reducing I/O
> cost, so a stronger RAID definitely will benefit PostgreSQL performance.

The 16x SATA drives should be great, provided you have a high performance
RAID adapter configured properly. You should be able to get 800MB/s of
sequential scan performance by using a card like the 3Ware 9550SX. I've
also heard that the Areca cards are good (how good?). Configuration of the
I/O must be validated though - I've seen as low as 25MB/s from a
misconfigured system.

>> For my own interests, I would like to at least cover the following bases:
>> 32 bit vs 64 bit vs 64 bit kernel + 32 bit user-space; data warehouse type
>> tests (data >> memory); and web prefs test (active data RAM)
>>
>
> Don't forget TPCC (data > memory, with intensive updates). So the benchmarks
> in my mind include TPCC, TPCH and TPCW.

I agree with Qingqing, though I think the OSTG DBT-3 (very similar to TPC-H)
is sufficient for data warehousing.

This is a fairly ambitious project - one problem I see is that MySQL may not
run all of these benchmarks, particularly the DBT-3. Also - would the rules
allow for mixing / matching pluggable features of the DBMS? Innodb versus
MyISAM?

- Luke