[PATCH] optimization of VALGRIND_MEMPOOL_* usage

From: "Ivan N(dot) Taranov" <i(dot)taranov(at)postgrespro(dot)ru>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: [PATCH] optimization of VALGRIND_MEMPOOL_* usage
Date: 2020-04-06 09:47:38
Message-ID: CAKqLMA8QFMVjAOMqj-fQRRQL31dgospJHbsKjd9cWZJt-Duzxg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Propose

Optimization of using the family of macros VALGRIND_MEMPOOL_*.

How?

- add the GetMemoryChunkCapacity method, which returns the size of
usable space in the chunk
- use GetMemoryChunkCapacity on VALGRIND_MEMPOOL_* calls
- call VALGRIND_MEMPOOL_CHANGED only for really changed chunks

*) Full patch code see in attachment
001-mem-chunk-capacity-and-repalloc-valgrind.patch

Why?

Under the valgrind control, the VALGRIND_MEMPOOL_CHANGE call works
very slooowly. With a large number of allocated memory chunks (a few
thousand and more) it is almost impossible to wait for the
program/test to done. This creates problems. During debugging and
auto-tests.

For example, below code is executed 90000ms on Core i7

for (int64 i = 0; i < 16000; ++i)
chunks[i] = palloc(64);

for (int64 i = 0; i < 16000; ++i)
chunks[i] = repalloc(chunks[i], 62);

With patch above - this code is executed 150ms.

*) Full extension code to demonstrate the problem see in attachment
valgrind_demo.tar.gz

An additional example

is the rum extension (https://github.com/postgrespro/rum).

To be able to perform the tests - need reduce the generate_series size
from 100K to 16K (https://github.com/postgrespro/rum/issues/76). But
under valgrind test execution remaining unacceptably sloow. Patch
above - completely solves the problem.

Attachment Content-Type Size
valgrind_demo.tar.gz application/gzip 1.2 KB
001-mem-chunk-capacity-and-repalloc-valgrind.patch text/x-patch 10.7 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro Horiguchi 2020-04-06 09:50:27 Re: [HACKERS] Restricting maximum keep segments by repslots
Previous Message Julien Rouhaud 2020-04-06 09:33:34 Re: WAL usage calculation patch