Re: Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

From: Sokolov Yura <funny(dot)falcon(at)postgrespro(dot)ru>
To: pgsql-hackers(at)postgresql(dot)org
Cc: pgsql-hackers-owner(at)postgresql(dot)org
Subject: Re: Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)
Date: 2017-08-10 12:30:25
Message-ID: 642da34694800dab801f04c62950ce8a@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

On 2017-07-31 18:56, Sokolov Yura wrote:
> Good day, every one.
>
> In attempt to improve performance of YCSB on zipfian distribution,
> it were found that significant time is spent in XidInMVCCSnapshot in
> scanning snapshot->xip array. While overall CPU time is not too
> noticable, it has measurable impact on scaleability.
>
> First I tried to sort snapshot->xip in GetSnapshotData, and search in a
> sorted array. But since snapshot->xip is not touched if no transaction
> contention occurs, sorting xip always is not best option.
>
> Then I sorted xip array on demand in XidInMVCCSnapshot only if
> search in snapshot->xip occurs (ie lazy sorting). It performs much
> better, but since it is O(NlogN), sort's execution time is noticable
> for large number of clients.
>
> Third approach (present in attached patch) is making hash table lazily
> on first search in xip array.
>
> Note: hash table is not built if number of "in-progress" xids is less
> than 60. Tests shows, there is no big benefit from doing so (at least
> on Intel Xeon).
>
> With regards,

Here is new, more correct, patch version, and results for pgbench with
zipfian distribution (50% read 50% write) (same to Alik's tests at
https://www.postgresql.org/message-id/BF3B6F54-68C3-417A-BFAB-FB4D66F2B410@postgrespro.ru
)

clients | master | hashsnap2 | hashsnap2_lwlock
--------+--------+-----------+------------------
10 | 203384 | 203813 | 204852
20 | 334344 | 334268 | 363510
40 | 228496 | 231777 | 383820
70 | 146892 | 148173 | 221326
110 | 99741 | 111580 | 157327
160 | 65257 | 81230 | 112028
230 | 38344 | 56790 | 77514
310 | 22355 | 39249 | 55907
400 | 13402 | 26899 | 39742
500 | 8382 | 17855 | 28362
650 | 5313 | 11450 | 17497
800 | 3352 | 7816 | 11030

(Note: I'm not quite sure, why my numbers for master are lower than
Alik's one. Probably, our test methodology differs a bit. Perhaps, it
is because I don't recreate tables between client number change, but
between branch switch).

With regards
--
Sokolov Yura aka funny_falcon
Postgres Professional: https://postgrespro.ru
The Russian Postgres Company

Attachment Content-Type Size
0001-Make-hash-table-for-xip-in-XidInMVCCSnapshot-v2.patch text/x-diff 11.0 KB
test_hashsnap_9.tar.gz application/x-gzip 94.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Chris Travers 2017-08-10 12:39:22 Re: Funny WAL corruption issue
Previous Message Ashutosh Sharma 2017-08-10 12:30:00 Re: pl/perl extension fails on Windows