8xIntel S3500 SSD in RAID10 on Dell H710p

From: Strahinja Kustudić <strahinjak(at)nordeus(dot)com>
To: "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: 8xIntel S3500 SSD in RAID10 on Dell H710p
Date: 2014-12-09 23:28:47
Message-ID: CADKbJJXupGHhwJvYHCgDJeNnm-DHjAQh5RjbTVgwWK1nza2ubQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

I have a beast of a Dell server with the following specifications:

- 4x Xeon E5-4657LV2 (48 cores total)
- 196GB RAM
- 2x SCSI 900GB in RAID1 (for the OS)
- 8x Intel S3500 SSD 240GB in RAID10
- H710p RAID controller, 1GB cache

Centos 6.6, RAID10 SSDs uses XFS (mkfs.xfs -i size=512 /dev/sdb).

Here are some relevant postgresql.conf settings:
shared_buffers = 8GB
work_mem = 64MB
maintenance_work_mem = 1GB
synchronous_commit = off
checkpoint_segments = 256
checkpoint_timeout = 10min
checkpoint_completion_target = 0.9
seq_page_cost = 1.0
effective_cache_size = 100GB

I ran some "fast" pgbench tests with 4, 6 and 8 drives in RAID10 and here
are the results:

time /usr/pgsql-9.1/bin/pgbench -U postgres -i -s 12000 pgbench # 292GB DB

4 drives6 drives8 drives105 min98 min94 min

/usr/pgsql-9.1/bin/pgbench -U postgres -c 96 -T 600 -N pgbench # Write
test

4 drives6 drives8 drives656774278073

/usr/pgsql-9.1/bin/pgbench -U postgres -c 96 -T 600 pgbench # Read/Write
test

4 drives6 drives8 drives365154747203

/usr/pgsql-9.1/bin/pgbench -U postgres -c 96 -T 600 -S pgbench # Read test

4 drives6 drives8 drives176282548228698

A few notes:

- I ran these tests only once, so take these number with reserve. I
didn't have the time to run them more times, because I had to test how the
server works with our app and it takes a considerable amount of time to run
them all.
- I wanted to use a bigger scale factor, but there is a bug in pgbench
with big scale factors.
- Postgres 9.1 was chosen, since the app which will run on this server
uses 9.1.
- These tests are with the H710p controller set to write-back (WB) and
with adaptive read ahead (ADRA). I ran a few tests with write-through (WT)
and no read ahead (NORA), but the results were worse.
- All tests were run using 96 clients as recommended on the pgbench wiki
page, but I'm sure I would get better results if I used 48 clients (1 for
each core), which I tried with the R/W test and got 7986 on 8 drives, which
is almost 800TPS better than with 96 clients.

Since our app is tied to the Postgres performance a lot, I'm currently
trying to optimize it. Do you have any suggestions what Postgres/system
settings I could try to tweak to increase performance? I have a feeling I
could get more performance out of this system.

Regards,
Strahinja

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Josh Berkus 2014-12-10 01:46:34 Re: Yet another abort-early plan disaster on 9.3
Previous Message Bruce Momjian 2014-12-09 20:43:37 Re: intel s3500 -- hot stuff