From: | "Ivan N(dot) Taranov" <i(dot)taranov(at)postgrespro(dot)ru> |
---|---|
To: | PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | [PATCH] optimization of VALGRIND_MEMPOOL_* usage |
Date: | 2020-04-06 09:47:38 |
Message-ID: | CAKqLMA8QFMVjAOMqj-fQRRQL31dgospJHbsKjd9cWZJt-Duzxg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Propose
Optimization of using the family of macros VALGRIND_MEMPOOL_*.
How?
- add the GetMemoryChunkCapacity method, which returns the size of
usable space in the chunk
- use GetMemoryChunkCapacity on VALGRIND_MEMPOOL_* calls
- call VALGRIND_MEMPOOL_CHANGED only for really changed chunks
*) Full patch code see in attachment
001-mem-chunk-capacity-and-repalloc-valgrind.patch
Why?
Under the valgrind control, the VALGRIND_MEMPOOL_CHANGE call works
very slooowly. With a large number of allocated memory chunks (a few
thousand and more) it is almost impossible to wait for the
program/test to done. This creates problems. During debugging and
auto-tests.
For example, below code is executed 90000ms on Core i7
for (int64 i = 0; i < 16000; ++i)
chunks[i] = palloc(64);
for (int64 i = 0; i < 16000; ++i)
chunks[i] = repalloc(chunks[i], 62);
With patch above - this code is executed 150ms.
*) Full extension code to demonstrate the problem see in attachment
valgrind_demo.tar.gz
An additional example
is the rum extension (https://github.com/postgrespro/rum)
To be able to perform the tests - need reduce the generate_series size
from 100K to 16K (https://github.com/postgrespro/rum/issues/76) But
under valgrind test execution remaining unacceptably sloow. Patch
above - completely solves the problem.
Attachment | Content-Type | Size |
---|---|---|
valgrind_demo.tar.gz | application/gzip | 1.2 KB |
001-mem-chunk-capacity-and-repalloc-valgrind.patch | text/x-patch | 10.7 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Kyotaro Horiguchi | 2020-04-06 09:50:27 | Re: [HACKERS] Restricting maximum keep segments by repslots |
Previous Message | Julien Rouhaud | 2020-04-06 09:33:34 | Re: WAL usage calculation patch |