Re: High CPU Utilization

On Mon, 16 Mar 2009, Joe Uhl wrote:

> Here is vmstat 1 30. We are under peak load right now so I can gather
> information from the real deal

Quite helpful, reformatting a bit and picking an informative section:

procs -----------memory---------- ---swap- ----io--- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 34 95048 11025880 56988 15020168 0 0 3852 160 3616 8614 11 1 6 82
3 25 95048 10996356 57044 15044796 0 0 7892 456 3126 7115 4 3 8 85
1 26 95048 10991692 57052 15050100 0 0 5188 176 2566 5976 3 2 12 83

This says that your server is spending all its time waiting for I/O,
actual CPU utilization is minimal. You're only achieving around 3-8MB/s
of random I/O. That's the reality of what your disk I/O subsystem is
capable of, regardless of what its sequential performance with dd looks
like. If you were to run a more complicated benchmark like bonnie++
instead, I'd bet that your "seeks/second" results are very low, even
though sequential read/write is fine.

The Perc5 controllers have a pretty bad reputation for performance on this
list, even in RAID10. Not much you can do about that beyond scrapping the
controller and getting a better one.

What you might do in order to reduce the total number of writes needed is
some standard postgresql.conf tuning; see
http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server

What you could do here is increase shared_buffers, checkpoint_segments,
and checkpoint_completion_target as described there. Having more buffers
dedicated to the database and having less checkpoints can result in less
random I/O showing up, as popular data pages will stay in RAM for longer
without getting written out so much.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD