Re: Safe memory allocation functions

Andres Freund wrote:
> On 2015-01-16 12:09:25 -0300, Alvaro Herrera wrote:
> > Robert Haas wrote:
> > > On Thu, Jan 15, 2015 at 10:57 AM, Alvaro Herrera
> > > <alvherre(at)2ndquadrant(dot)com> wrote:
> >
> > > >> I do think that "safe" is the wrong suffix. Maybe palloc_soft_fail()
> > > >> or palloc_null() or palloc_no_oom() or palloc_unsafe().
> > > >
> > > > I liked palloc_noerror() better myself FWIW.
> > >
> > > I don't care for noerror() because it probably still will error in
> > > some circumstances; just not for OOM.
> >
> > Yes, but that seems fine to me. We have other functions with "noerror"
> > flags, and they can still fail under some circumstances -- just not if
> > the error is the most commonly considered scenario in which they fail.
>
> We rely on palloc erroring out on large allocations in a couple places
> as a crosscheck. I don't think this argument holds much water.

I don't understand what that has to do with it. Surely we're not going
to have palloc_noerror() not error out when presented with a huge
allocation. My point is just that the "noerror" bit in palloc_noerror()
means that it doesn't fail in OOM, and that there are other causes for
it to error.

One thought I just had is that we also have MemoryContextAllocHuge; are
we going to consider a mixture of both things in the future, i.e. allow
huge allocations to return NULL when OOM? It sounds a bit useless
currently, but it doesn't seem extremely far-fetched that we will need
further flags in the future. (Or, perhaps, we will want to have code
that retries a Huge allocation that returns NULL with a smaller size,
just in case it does work.) Maybe what we need is to turn these things
into flags to a new generic function. Furthermore, I question whether
we really need a "palloc" variant -- I mean, can we live with just the
MemoryContext API instead? As with the "Huge" variant (which does not
have a corresponding palloc equivalent), possible use cases seem very
limited so there's probably not much point in providing a shortcut.

So how about something like

#define ALLOCFLAG_HUGE 0x01
#define ALLOCFLAG_NO_ERROR_ON_OOM 0x02
void *
MemoryContextAllocFlags(MemoryContext context, Size size, int flags);

and perhaps even

#define MemoryContextAllocHuge(cxt, sz) \
MemoryContextAllocFlags(cxt, sz, ALLOCFLAG_HUGE)
for source-level compatibility.

(Now we all agree that palloc() itself is a very hot spot and shouldn't
be touched at all. I don't think these new functions are used as commonly
as that, so the fact that they are slightly slower shouldn't be too
troublesome.)

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services