Re: pg 8.1.3, AIX, huge box, painfully slow.

Gavin,

On 4/7/06 2:24 PM, "Gavin Hamill" <gdh(at)laterooms(dot)com> wrote:

> I did look into the specs of the system, and the memory bw on the
> pSeries was /much/ greater than the Xeon - it's one of the things that
> really pushed me towards it in the end. I forget the figures, but it
> was 3 or 4 times greater.

From the literature at:

http://www-03.ibm.com/servers/eserver/pseries/hardware/midrange/p650_desc.ht
ml

³The pSeries 650 features a peak aggregate memory to L3 cache bandwidth of
25.6GB/second for an 8way configuration. In addition, aggregate I/O
bandwidth is up to 16GB/second. The result is a remarkable combination of
system architecture, speed and power that delivers efficient and
cost-effective data sharing and application throughput.²

That¹s a total of 25.6GB/s for 8 CPUs, or 3.2GB/s per CPU. 3GHz P4 Xeons
typically have an 800MHz memory bus with double the speed at 6.4GB/s result
(800MHz x 8 bytes per L2 cache line = 6.4GB/s). Furthermore, the speed at
which the P4 Xeon can load data into L2 cache from memory is matched to the
bus because the L2 cache line width is 8 bytes wide and can stream data to
L2 at full bus speed.

That said, I find typical memory bandwidth for the P4 in applications is
limited at about 2GB/s. See here for more detail:
http://www.cs.virginia.edu/stream/standard/Bandwidth.html

In fact, looking at the results there, the IBM 650m2 only gets 6GB/s on all
8 CPUs. I wouldn¹t be surprised if the strange L3 cache architecture of the
IBM 650 is holding it back from streaming memory access efficiently.

Whether this has anything to do with your problem or not, I have no idea!

- Luke