Re: effective_io_concurrency

From: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
To: Peter Geoghegan <peter(at)2ndquadrant(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: effective_io_concurrency
Date: 2012-09-01 02:20:44
Message-ID: CAMkU=1zNT5qaHWuJGpw9Xqm0GGpEb4LC2EtqxCCs8bjct8JsUw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Aug 30, 2012 at 1:25 PM, Peter Geoghegan <peter(at)2ndquadrant(dot)com> wrote:

> Just how helpful is effective_io_concurrency? Did someone produce a
> benchmark at some point?

Attached is a benchmark I put together a while ago.

I don't know how close to "real world" it might be. I haven't seen it
in the wild, but I'm anticipating I will see something like it soon.

The benefit is of course reduced if you were to apply high levels of
-c $clients, but cases where -c 1 are frequent in data mining and
such.

Obviously it would be better to cluster the giant table on "abalance",
but that might be hard to do and then hard to maintain.

Size of generated database is about 70GB.

Server had 16GB of RAM. It is a virtual server, and I don't know the
real hardware, but am told it has 8 spindles.
(if the range used in the query is increased from +100 to +1000, even
more speed up is available)

effective_io_concurrency tps
0 2.06273064
1 2.1693092
2 4.11726948
3 5.90785352
4 6.65748384
5 7.58297556
6 8.36130404
7 8.86561116
8 9.2673546
9 9.57076168
10 9.8558758
11 10.11641752
12 10.316673
13 10.46953468
14 10.65962516
15 10.76328636
16 10.86442376
17 10.96362168
18 11.04371008
19 11.19470171
20 11.30110867
21 11.39553967
22 11.45420263
23 11.54764725
24 11.61949146
25 11.65659225
26 11.68992392
27 11.75944667
28 11.7456135
29 11.80111779
30 11.72897188
31 11.7210945
32 11.73292504
33 11.734458
34 11.75195196
35 11.79079175
36 11.73687979
37 11.79583758
38 11.75879063
39 11.77868596
40 11.74685896
41 11.76294508
42 11.7213265
43 11.68458158
44 11.71036729
45 11.72728229
46 11.72063796
47 11.80322429
48 11.83563058
49 11.81916996
50 11.73395892

Cheers,

Jeff

Attachment Content-Type Size
bitmap.sql application/octet-stream 131 bytes
bench.sh application/x-sh 191 bytes
setup.sql application/octet-stream 217 bytes

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2012-09-01 04:00:39 too much pgbench init output
Previous Message Andrew Dunstan 2012-09-01 00:35:48 Re: _USE_32BIT_TIME_T Patch