Re: Threaded PosgreSQL server

On Fri, 8 Feb 2002, Tom Lane wrote:

> I have a sneaking feeling that what you are going to come up with is a
> multi-megabyte patch to convert CurrentMemoryContext into a non-global,
> which will require changing the parameter list of damn near every
> routine in the backend.

While working with 7.0.2, I changed the call signature on only about 10
functions. In the MemoryContext example,
MemorycontextSwitchTo(<Any>MemoryContext) turned into
MemoryContextSwitchTo(GetEnv()-><Any>MemoryContext). You may be able
to do this with a #define. While profiling the
code, this actually had very little impact on CPU resources. There were
some hotspots where it made more sense to pass the global environment to
the function but the list is small.

>
> Personally I will vote for rejecting such a patch, as it will uglify the
> code (and break nearly all existing user-written extension functions)
> far more than is justified by what it accomplishes: exactly zero, in
> terms of near-term usefulness.

I don't think that user functions need be broken. As long as they use
palloc, a recompile may be all that is needed.

>
> I think what's more interesting to discuss at this stage is the
> considerations I alluded to before: what are we going to do with the
> caches and other potentially-sharable datastructures? Without a
> credible design for those issues, there is no point in sweating the
> small-but-annoying stuff.

As far as caches go, I punted on sharing. Controlling access to the cache
hash tables looked like alot of work and I thought the contention for this
resource would be high. So I had each thread build separate cache
structures. The one difference was I had the original cache build occur
from memory rather than the file pg_internal.init. So when the first
thread for a particular db is built, the cache structures are built in
system memory and copied into the appropriate MemoryContext. Each
subsequent cache for the db is copied from main memory at thread build.

One place where sharing worked great was the file manager. I modified
md.c to share Vfd's. I made the maximum number of threads that could share
one Vfd configurable so that the number of Vfds created and the contention
to those Vfd's could be balanced.

It seems obvious to me that we need to thread slowly and softly into this
area so I promise I will not to spend a ton of time mangling the whole CVS
tree,
that most definitely, would be a waste of everybody's time. I think I can
find an example area that will be a small patch and submit it for review.
Hopefully this can get the ball rolling.

Myron
mkscott(at)sacadia(dot)com