Re: Improving the memory allocator

On Tuesday, April 26, 2011 01:24:20 AM Robert Haas wrote:
> On Mon, Apr 25, 2011 at 5:45 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > [ lots of awesome test results ]
>
> Very interesting work. I think it's interesting that there is so much
> allocation happening inside MessageContext; I barely knew that
> existed, let alone that it was a hotspot for memory allocation.
Yes, I was surprised as well. I haven't checked yet what the callsites for
that are.
As I just wrote to TOm the hotspots are wildly different if you use slightly
more complex statements. Which doesn't mean its not worthy of improvement ;)

> I
> think it would be interesting to break out the calls to new_list() by
> caller. It's been vaguely bothering me for a long time that we store
> so many things in using List structures. That is not a particularly
> efficient representation, because a List of N nodes requires 2N+1
> memory allocations - N nodes, N ListCell objects, and the List object
> itself. It might be worth experimenting with some kind of array/list
> hybridy thing, like this:
> ...
While I aggree that that might be beneficial I think in this case nearly all
of the lists are of 1 element length because its callers seem to look mostly
like:
{{{
List *
lappend(List *list, void *datum)
{
Assert(IsPointerList(list));

if (list == NIL)
list = new_list(T_List);
else
new_tail_cell(list);

lfirst(list->tail) = datum;
check_list_invariants(list);
return list;
}
}}}

Hm. Seems worthy of some further investigation...

Thanks,

Andres