PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

From: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
To: pgsql-hackers(at)postgreSQL(dot)org
Subject: PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)
Date: 2017-08-14 00:35:42
Message-ID: fe2e6aba-991e-27e3-eb06-1429db17e200@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Attached is a rebased version of the Generational context, originally
submitted with SlabContext (which was already committed into Pg 10).

The main change is that I've abandoned the pattern of defining a Data
structure and then a pointer typedef, i.e.

typedef struct GenerationContextData { ... } GenerationContextData;
typedef struct GenerationContextData *GenerationContext;

Now it's just

typedef struct GenerationContext { ... } GenerationContext;

mostly because SlabContext was committed like that, and because Andres
was complaining about this code pattern ;-)

Otherwise the design is the same as repeatedly discussed before.

To show that this is still valuable change (even after SlabContext and
adding doubly-linked list to AllocSet), I've repeated the test done by
Andres in [1] using the test case described in [2], that is

-- generate data
SELECT COUNT(*) FROM (SELECT test1()
FROM generate_series(1, 50000)) foo;

-- benchmark (measure time and VmPeak)
SELECT COUNT(*) FROM (SELECT *
FROM pg_logical_slot_get_changes('test', NULL,
NULL, 'include-xids', '0')) foo;

with different values passed to the first step (instead of the 50000).
The VmPeak numbers look like this:

N master patched
--------------------------------------
100000 1155220 kB 361604 kB
200000 2020668 kB 434060 kB
300000 2890236 kB 502452 kB
400000 3751592 kB 570816 kB
500000 4621124 kB 639168 kB

and the timing (on assert-enabled build):

N master patched
--------------------------------------
100000 1103.182 ms 412.734 ms
200000 2216.711 ms 820.438 ms
300000 3320.095 ms 1223.576 ms
400000 4584.919 ms 1621.261 ms
500000 5590.444 ms 2113.820 ms

So it seems it's still a significant improvement, both in terms of
memory usage and timing. Admittedly, this is a single test, so ideas of
other useful test cases are welcome.

regards

[1]
https://www.postgresql.org/message-id/20170227111732.vrx5v72ighehwpkf%40alap3.anarazel.de

[2]
https://www.postgresql.org/message-id/20160706185502.1426.28143%40wrigleys.postgresql.org

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment Content-Type Size
0001-Generational-memory-allocator.patch text/x-patch 31.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message AP 2017-08-14 00:40:36 Re: pgsql 10: hash indexes testing
Previous Message Andres Freund 2017-08-14 00:32:21 Re: POC: Sharing record typmods between backends