Re: [HACKERS] Modules

Mattias Kregert writes:
> David Gould wrote:
...
> > You need to be able to
> > - add one or more type descriptions types table
[big list of stuff to do deleted ]
> > - do all the above within a particular schema
> >
> > You may also need to create and populate data tables, rules, defaults, etc
> > required by the implementation of the new type.
>
> All this would be done by the init function in the module you load.
> What we need is a set of functions callable by modules, like
> module_register_type(name, descr, func*, textin*, textout*, whatever
> ...)
> module_register_smgr(name, descr, .....)
> module_register_command(....

Ok, now you are requiring the module to handle all this in C. How does it
register a type, a default, a rule, a column, functions, etc?

Having thought about that, consider that currently all this can already be
done using SQL. postgreSQL is a relational database system. One of the
prime attributes of relational systems is that they are reflexive. That is,
you can use SQL to query and update the system catalogs that define the
characteristics of the the system.

By and large, all the tasks I mentioned previously can be done using SQL and
taking advantage of the high semantic level and power of the complete SQL
system. Given that we already have a perfectly good high level mechanism, I
just don't see any advantage to adding a bunch of low level APIs to
duplicate existing functionality.

> Casts would be done by converting to a common format (text) and then to
> the desired type. Use textin/textout. No special cast functions would
> have to exist. Why doesn't it work this way already??? Would not that
> solve all casting problems?

No. It is usable in some cases as an implementation of a defined cast, but
you still need defined casts. Think about these problems:

First, there is a significant performance penalty. Think about a query
like:

select * from huge_table where account_balance > 1000.

The textout -> textin approach would be far slower than the current direct
int to float cast.

Second, how do you restrict the system to sensible casts or enforce a
meaningful order of attempted casts.

create type yen based float;
create type centigrade based float;

Would you allow?

select yen * centigrade from market_data, weather_data
where market_data.date = weather_data.date;

Even though the types 'yen' and 'centigrade' are implemented by float this
leaves open a few important questions:

- what is the type of the result?
- what could the result possibly mean?

Third you still can't do casts for many types:

create type motion_picture (arrayof jpeg) ...

select motion_picture * 10 from films...

There is no useful cast possible here.

> > To unload a type requires undoing all the above. But there is a wrinkle: first
> > you have to check if there are any dependancies. That is, if the user has
> > created a table with one of the new types, you have to drop that table
> > (including column defs, indexes, rules, triggers, defaults etc) before
> > you can drop the type. Of course the user may not want to drop their tables
> > which brings us to the the next problem.
>
> Dependencies are checked by the OS kernel when you try to unload
> modules.
> You cannot unload slhc without first unloading ppp, for example. What's
> the
> difference?

I could have several million objects that might use that type. I cannot
do anything with them without the type definition. Not even delete them.

> If you have Mod4X running with /dev/dsp opened, then you can't unload
> the sound driver, because it is in use, and you cannot unload a.out
> module
> modules and so on... This would not be different in a SQL server.

But it is very different. SQL servers are much more complex than OS kernels.
Having spent a number of years maintaining the OS kernel in a SQL engine
that was originally intended to run on bare hardware, I can tell you that
that kernel was less than 10% of the complete SQL engine.

> If you have a cursor open, accessing IP types, then you cannot unload
> the IP-types module. Close the cursor, and you can unload the module if
> you want to.
> You don't have to drop tables containing new types just because you
> unload
> the module. If you want to SELECT from it, then that module would be
> loaded
> automagically when it is needed.

Ahha, I start to understand. You are saying 'module' and meaning 'loadable
object file of functions'. Given that this is what you mean, we already
handle this.

What I took you to mean by 'module' was the SCHEMA defined to make the
functions useful, and the functions.

> > Just to belabor this, it is perfectly reasonable to add a set of types and
> > functions that have no 'C' implementation. The 'loadable module' analogy
> > misses a lot of the real requirements.
>
> Why would someone want a type without implementation?

Why should a type with no C functions fail to have an implementation? Right
now every table is also a type.

Many types are based on a extending an existing type, or are composites. Is
there some reason not to define the implementation (if any) in SQL?

--

I understand that modularity is good. I am asserting the postgreSQL is a
very modular and extendable system right now. There are mechanisms to add
just about any sort of extension you want. Very little is hard coded in
the core.

I think this discussion got started because someone wanted to remove the
ip and mac and money types. This is a mere matter of the current packaging,
as there is no reason to for them to be in or out except that historically
the system used some of these types before the extendibility was finished
so they went in the core code.

I don't think it matters much whether any particular type is part of the
core or not, so feel free to pull them out. Do package them up to
install in the normal way that extensions are supposed to install and
retest everything. Don't _add_ a whole new way to do the same kinds
of extensibilty that we _already_ do. Just use the mechanisms that already
exist.

This discussion has gone on too long as others are starting to point out,
so I am happy to take it to private mail if you wish to continue.

-dg

David Gould dg(at)illustra(dot)com 510.628.3783 or 510.305.9468
Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612
"Of course, someone who knows more about this will correct me if I'm wrong,
and someone who knows less will correct me if I'm right."
--David Palmer (palmer(at)tybalt(dot)caltech(dot)edu)