Re: Extension Templates S03E11

On Tue, 2013-11-26 at 01:37 +0200, Heikki Linnakangas wrote:
> I want to be able to download extension.zip from pgxn.org, and then
> install it on a server. I want to be able to install it the traditional
> way, by unzipping it to the filesystem, or via libpq by using this new
> feature.

I see where you're coming from, but after some thought, and looking at
the patch, I think we really do want a catalog representation for (at
least some) extensions.

Dealing with files is a conceptual mismatch that will never be as easy
and coherent as something that database manages and understands.
Replication, backup, and Postgres-as-a-service providers are clear
examples, and I just don't see a file-based approach solving those
problems.

One example is tablespaces, which are a constant source of
administrative pain. That's inherent to tablespaces, just like it's
inherent to native shared libraries, and we can't do much about them.

But bringing more of an extension into the catalog can be done, and I
think we'll see big benefits from that.

Imagine something like (this comes from an in-person conversation with
Dimitri a while ago; hopefully I'm not misrepresenting his vision):

=# select pgxn_install_template('myextension');

or even:

=# select pgxn_update_all_templates();

That is much closer to what modern language environments do -- ruby,
python, go, and haskell all have a language-managed extension service
independent of the OS packaging system and don't require more privileges
or access than running the language.

That being said, there some things about in-catalog templates that need
some more thought:

1. If someone does want their OS to install extensions for them (e.g.
the contrib package), how should that be done? This usually works fine
with the aforementioned languages, because installation is still just
dropping files in the right place. Postgres is different, because to put
something in the catalog, we need a running server, which is awkward for
a packaging system to do.

2. When 9.4 gets released, we need some solid advice for extension
authors. If they have a native shared library, I assume we just tell
them to keep using the file-based templates. But if they have a SQL-only
extension, do we tell them to port to the in-catalog templates? What if
they port to in-catalog templates, and then decide they just want to
optimize one function by writing it in native code? Do they have to port
back? What should the authors of SQL-only extensions distribute on PGXN?
Should there be a migration period where they offer both kinds of
templates until they drop support for 9.3?
a. Some extensions have quite a few .sql files. It seems awkward to
just cat them all into one giant SQL query. Not a rational problem, but
it would bother me a little to tell people to squash their
otherwise-organized functions into a giant blob.

3. What do we do about native shared libraries? Ultimately, I imagine
that we should handle these similarly to tablespaces: have a real
database object with an OID that extensions or functions can depend on,
and create a symlink (with the OID as the link name) that points to the
real file on disk. We could also export some new symbols like the shared
library name and version for better error checking.

4. Do we live with both file-based and catalog-based templates
forever? I guess probably so, because the file-based templates probably
are a little better for contrib itself (because the complaints about
relying on OS packaging don't apply as strongly, if at all).

Regards,
Jeff Davis