Re: type design guidance needed

Hi Brook,
It seems to me that the answer depends on how much effort you want to
put in to it.

Brook Milligan wrote:
>
> I am working on designing some new datatypes and could use some
> guidance.

> It seems that there are several alternative approaches, and I am
> seeking some guidance from the wizards here who have some
> understanding of the backend internals, performance tradeoffs, and
> such issues.
>
> Possible solutions:
>
> 1. Store the data and all the scale parameters within the type.
>

Probably the easiest solution, but it might leave a furry taste in your
mouth.

>
> 2. Store only the data and a reference to a compiled-in data table
> holding the scale parameters.
>

If you can fix all the parameters at compile time this is a good
solution. Don't forget that the code for the type is going to
dynamically linked into the backend, so "compile time" can happen on the
fly. You can write an external script to update your function's source
code with the new data, compile the updated source and relink the new
executable. This is of course a hack. If you really want to do this on
the fly you would need to be sure that simultaneously executing
backends, which might be linked to different versions of the library,
always do consistent things with the datatypes. Also, you might want to
check out what exactly happens when a backend links new symbols over old
ones.

I have actually done this in a situation where I wanted to load a bunch
of values in to a database, do some analysis, change parameters in
backend functions, and repeat. I was only using a single backend at a
time, and I closed the backend between relinks. It is easier than it
sounds, the only part of the backend that you need to understand is the
function manager and dynamic loader.

The major advantage of this approach is that you are not hacking the
backend, and your code might actually continue to work across release
versions.

>
> 3. Store only the data and a reference to a new system table (or
> tables) holding the scale parameters.
>
>

This solution is probably the neatest, and in the long term the most
robust. However it might also involve the most effort. Depending on how
you are using the datatypes, you will probably want to cache the
information that you are storing in "system tables" locally in each
backend. Basically this means allocating an array to a statically
allocated pointer in your code, and populating the array with the data
from the system table the first time that you use it. You also need to
write a trigger that will invalidate the caches in all backends in the
event that the system table is updated. There is already a lot of
caching code in the backend, and a system for cache invalidation. I
expect that you would end up modifying or copying that.

Another point about writing internal backend code is that you end up
writing to changing interfaces. You can expect your custom
modifications to break with every release. The new function manager,
which is a much needed and neatly executed improvement, broke all of my
code. This would be a major consideration if you had to support the code
across several releases. Of course if the code is going to be generally
useful, and the person who pays you is amenable, you can always submit
the modifications as patches.

>
> Cheers,
> Brook