Re: GiST memory allocation

Neil Conway <neilc(at)samurai(dot)com> writes:
> AFAICS, GiST doesn't take any advantage of the palloc() infrastructure
> beyond treating palloc() as a better malloc().

This is pretty much true of all the index AMs, I think. I looked
briefly at using a short-term memory context in the btree code, but
gave it up after seeing that many of the allocations have to survive
from one index_getnext call to the next. It didn't seem that there
would be much net gain in notational simplicity.

> (the observation here is that 99% of the allocations done by
> gist.c are for internal use only -- we rarely allocate anything that
> needs to live longer than the current GiST API call).

You sure about that? In btree quite a lot of the allocs need to survive
across the current scan.

> One relatively easy fix would be to create a per-backend "GiST context"
> as a static variable in gist.c.
> ...
> One downside is that the GiST API won't be reentrant, but I don't think
> that's a major problem.

I think this is not acceptable. It certainly wouldn't be acceptable for
btree --- you couldn't even use a PL-language function as an index
operator, because the PL itself would need to do system catalog accesses
that could result in re-entrant btree scans. If you got away with it
for GiST it would only be because GiST is a stepchild that the system
doesn't depend on. That doesn't sound like the direction to go in.

You could look at creating per-scan and/or per-tuple-cycle memory
contexts during gistbeginscan, keeping pointers to them in the indexscan
structure. However this only works for scans --- not sure if there are
any data structures that are common to scans and the non-scan operations.

regards, tom lane