Re: [HACKERS] Postgres Speed or lack thereof

>
> Tom Lane wrote:
> >
> > > Note that our mmgr adds 16 bytes to each allocation
> > > (+ some bytes in malloc) - a great overhead, yes?
> >
> > Youch ... for a list-node-sized request, that's a lot of overhead.
>
> And lists are very often used by planner and - never pfreed.
>
> > Getting this right is probably going to take some thought and work,
> > but it looks worthwhile from a performance standpoint. Maybe for 6.6?
>
> Yes - I have to return to MVCC stuff...
> So, I consider my exercizes with mmgr as vacation from MVCC -:)
>
> > > It shows that we should get rid of system malloc/free and do
> > > all things in mmgr itself - this would allow us much faster
> > > free memory contexts at statement/transaction end.
> >
> > I don't think we can or should stop using malloc(), but we can
> > ask it for large blocks and do our own allocations inside those
> > blocks --- was that what you meant?
>
> No. We could ask brk() for large blocks.
> The problem is where to handle dynamic allocations.
> As long as they are handled by malloc we can't put
> them in proper blocks of current memory context.
> But having our own handling malloc would become useless.
>
> Vadim

We could use 4 methods for dynamic allocation:

1) malloc/free - for persistent storage allocation

2) palloc/pfree - for storage belonging to some context and
which we can keep track of and free explicitly

3) fast_palloc - for storage which impossible, too difficult or too
expensive to keep track of. This storage should be allocated with
fast and simple inline code from bigger chunks allocated with palloc.
This storage would never freed explicitly, so that code could be
simple and fast, but the big chunks would be freed automatically at
the end of the transaction.

4) fast_talloc - we could introduce a `tuple' context handled like
fast_palloc for storage used only while processing one tuple.
This storage could be fast allocated from few big chunks allocated
with palloc and freed explicitly after the tuple has been processed.
This could avoid the overhead of many malloc/palloc while reducing
the overall memory usage for transaction which process many rows.
The total cost per tuple could be one palloc and one pfree.
We could also simply reuse the chunks for every tuple and pfree them
only at the end of the transaction. This would cost one palloc/pfree
per transaction.

This would require revising the code and changing palloc/pfree with the new
functions where appropriate, but this could be done gradually because the
old palloc/pfree are always safe.

--
Massimo Dal Zotto

+----------------------------------------------------------------------+
| Massimo Dal Zotto email: dz(at)cs(dot)unitn(dot)it |
| Via Marconi, 141 phone: ++39-0461534251 |
| 38057 Pergine Valsugana (TN) www: http://www.cs.unitn.it/~dz/ |
| Italy pgp: finger dz(at)tango(dot)cs(dot)unitn(dot)it |
+----------------------------------------------------------------------+