Re: Safe memory allocation functions

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Safe memory allocation functions
Date: 2015-01-30 06:10:46
Message-ID: CAB7nPqQka2dMENhRym3+JnFbgFFYwQz6i5mh1GqpxFC4zQoc2g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I wrote:
> Yes, this refactoring was good for testing actually...
Oops, I have been too hasty when sending previous patch, there was a
bug related to huge allocations. Patch correcting this bug is
attached.

Attached are as well two things I have used to test the new API:
- A hack refactoring the existing routines MemoryContextAlloc* to use
the extended API
- An extension with a function doing a direct call to the extended API
able to control the flags used:
CREATE FUNCTION blackhole_palloc(size bigint,
is_huge bool,
is_no_oom bool,
is_zero bool,
is_zero_aligned bool)

Here are some tests done on a small box of 384MB with direct calls of
the extended API:
=# create extension blackhole ;
CREATE EXTENSION
-- failure for normal allocation because size >= 1GB
=# select blackhole_palloc(1024 * 1024 * 1024, false, false, false, false);
ERROR: XX000: invalid memory alloc request size 1073741824
LOCATION: MemoryContextAllocExtended, mcxt.c:628
-- Failure of OOM because normal allocation can be done, but no memory
=# select blackhole_palloc(1024 * 1024 * 1024 - 1, false, false, false, false);
ERROR: 53200: out of memory
DETAIL: Failed on request of size 1073741823.
LOCATION: MemoryContextAllocExtended, mcxt.c:639
-- No failure, bypassing OOM error
=# select blackhole_palloc(1024 * 1024 * 1024 - 1, false, true, false, false);
blackhole_palloc
------------------
null
(1 row)
-- Huge allocation, no error because OOM error is bypassed
=# select blackhole_palloc(1024 * 1024 * 1024, true, true, false, false);
blackhole_palloc
------------------
null
(1 row)
-- OOM error, huge allocation failure
=# select blackhole_palloc(1024 * 1024 * 1024, true, false, false, false);
ERROR: 53200: out of memory
DETAIL: Failed on request of size 1073741824.
LOCATION: MemoryContextAllocExtended, mcxt.c:639
-- Assertion failure, zero and zero aligned cannot be called at the same time
=# select blackhole_palloc(1024 * 1024, false, false, true, true);
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
--
Michael

Attachment Content-Type Size
0001-Create-MemoryContextAllocExtended-central-routine-fo.patch text/x-diff 3.8 KB
0002-Small-hack-to-test-extended-function-for-mcxt.patch text/x-diff 2.2 KB
blackhole.tar.gz application/x-gzip 1.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2015-01-30 06:20:36 Re: jsonb, unicode escapes and escaped backslashes
Previous Message Tom Lane 2015-01-30 06:04:06 Re: jsonb, unicode escapes and escaped backslashes