d88a45e680327e0b22a34020d8f78252 - Re: MongoDB 3.2 beating Postgres 9.5.1?

Hi,
I recently test YCSB benchmark too.
But contrary to my expectation, PG (9.5) is slower than MongoDB 3.2.
Paul said that making table with no logging option improved the performance,
and it might be equal to MongoDB's behavior.
But in MongoDB documentation, it writes journal log too.
So I think turning off no logging option is not fair.
Am I wrong about MongoDB's behavior?

[http://webmail.bitnine.co.kr/images/?attachID=cmVjcnVpdHxiaXRuaW5lX0NJXzYucG5n&cid=381831a483be2a2d269bb9df50ec4d49&date=20140911&adminId=andrewpark]
(C)Bitnine, Kisung Kim, Ph.D
https://sites.google.com/site/kisungresearch/
E-mail : kskim(at)bitnine(dot)net<mailto:kskim(at)bitnine(dot)net>
Office phone : 070-4800-5890, 408-606-8602
US Mobile phone : 408-805-2192

2016-03-19 5:05 GMT+09:00 <pbj(at)cmicdo(dot)com<mailto:pbj(at)cmicdo(dot)com>>:

On Tuesday, March 15, 2016 7:39 PM, "pbj(at)cmicdo(dot)com<mailto:pbj(at)cmicdo(dot)com>" <pbj(at)cmicdo(dot)com<mailto:pbj(at)cmicdo(dot)com>> wrote:

> Your results are close enough to mine, I think, to prove the point.
> And, I agree that the EDB benchmark is not necessary reflective of a
> real-world scenario.
>
> However, the cache I'm referring to is PG's shared_buffer cache.
> You can see the first run of the select causing a lot of disk reads.
> The second identical run, reads purely from shared_buffers.
>
> What I don't understand is, why does a slightly different select from
> the *same* table during the same session cause shared_buffers to be
> blown out and re-read??
>
> I will see if I can try YCSB next week (I'm in workshops all week...)
>
> Thanks!

I was able to try YCSB today on both PG 9.5.1 and Mongo 3.2. At first, PG
was running 4 times slower than Mongo. Then I remembered about unlogged
tables (which I think is the way Mongo is all the time.), and remade
the PG table as UNLOGGED. In a 50/50 read/update test over 1M records,
PG ran in 0.62 of the time of Mongo.

PG Load:
--------
[OVERALL], RunTime(ms), 104507.0
[OVERALL], Throughput(ops/sec), 9568.737022400413
[CLEANUP], Operations, 1.0
[CLEANUP], AverageLatency(us), 293.0
[CLEANUP], MinLatency(us), 293.0
[CLEANUP], MaxLatency(us), 293.0
[CLEANUP], 95thPercentileLatency(us), 293.0
[CLEANUP], 99thPercentileLatency(us), 293.0
[INSERT], Operations, 1000000.0
[INSERT], AverageLatency(us), 101.329235
[INSERT], MinLatency(us), 88.0
[INSERT], MaxLatency(us), 252543.0
[INSERT], 95thPercentileLatency(us), 121.0
[INSERT], 99thPercentileLatency(us), 141.0
[INSERT], Return=OK, 1000000

PG Run:
-------
[OVERALL], RunTime(ms), 92763.0
[OVERALL], Throughput(ops/sec), 10780.16019318047
[READ], Operations, 499922.0
[READ], AverageLatency(us), 79.1722428698877
[READ], MinLatency(us), 69.0
[READ], MaxLatency(us), 19935.0
[READ], 95thPercentileLatency(us), 94.0
[READ], 99thPercentileLatency(us), 112.0
[READ], Return=OK, 499922
[CLEANUP], Operations, 1.0
[CLEANUP], AverageLatency(us), 222.0
[CLEANUP], MinLatency(us), 222.0
[CLEANUP], MaxLatency(us), 222.0
[CLEANUP], 95thPercentileLatency(us), 222.0
[CLEANUP], 99thPercentileLatency(us), 222.0
[UPDATE], Operations, 500078.0
[UPDATE], AverageLatency(us), 98.96430156895525
[UPDATE], MinLatency(us), 83.0
[UPDATE], MaxLatency(us), 26655.0
[UPDATE], 95thPercentileLatency(us), 127.0
[UPDATE], 99thPercentileLatency(us), 158.0
[UPDATE], Return=OK, 500078

Mongo Load:
-----------
[OVERALL], RunTime(ms), 133308.0
[OVERALL], Throughput(ops/sec), 7501.425270801452
[CLEANUP], Operations, 1.0
[CLEANUP], AverageLatency(us), 1822.0
[CLEANUP], MinLatency(us), 1822.0
[CLEANUP], MaxLatency(us), 1822.0
[CLEANUP], 95thPercentileLatency(us), 1822.0
[CLEANUP], 99thPercentileLatency(us), 1822.0
[INSERT], Operations, 1000000.0
[INSERT], AverageLatency(us), 130.830678
[INSERT], MinLatency(us), 90.0
[INSERT], MaxLatency(us), 7147519.0
[INSERT], 95thPercentileLatency(us), 159.0
[INSERT], 99thPercentileLatency(us), 226.0
[INSERT], Return=OK, 1000000

Mongo Run:
---------
[OVERALL], RunTime(ms), 149150.0
[OVERALL], Throughput(ops/sec), 6704.65973851827
[READ], Operations, 500837.0
[READ], AverageLatency(us), 98.13153980237084
[READ], MinLatency(us), 69.0
[READ], MaxLatency(us), 28271.0
[READ], 95thPercentileLatency(us), 166.0
[READ], 99thPercentileLatency(us), 186.0
[READ], Return=OK, 500837
[CLEANUP], Operations, 1.0
[CLEANUP], AverageLatency(us), 2387.0
[CLEANUP], MinLatency(us), 2386.0
[CLEANUP], MaxLatency(us), 2387.0
[CLEANUP], 95thPercentileLatency(us), 2387.0
[CLEANUP], 99thPercentileLatency(us), 2387.0
[UPDATE], Operations, 499163.0
[UPDATE], AverageLatency(us), 195.21505600375028
[UPDATE], MinLatency(us), 118.0
[UPDATE], MaxLatency(us), 4513791.0
[UPDATE], 95thPercentileLatency(us), 211.0
[UPDATE], 99thPercentileLatency(us), 252.0
[UPDATE], Return=OK, 499163

>
>
> On Monday, March 14, 2016 3:34 AM, Dmitry Dolgov <9erthalion6(at)gmail(dot)com<mailto:9erthalion6(at)gmail(dot)com>> wrote:
>
>
> Hi, Paul
>
> I agree with Oleg, EDB benchmarks are strange sometimes. I did the same benchmarks several months ago. I never noticed the cache influence back then, so I tried to reproduce your situation now (on a 5*10^6 records although). I started to play with db cache (using `echo 3 > /proc/sys/vm/drop_cache`), and I see difference in time execution for two subsequent queries, but `explain` info are almost identical, e.g. `shared hit & read`:
>
> ....