From 7fb3f8971c4c573903b1d08bcbeb126e31a6544c Mon Sep 17 00:00:00 2001 From: Chapman Flack Date: Mon, 21 Feb 2022 20:59:32 -0500 Subject: [PATCH v2 2/4] Update PL handler implementation docs The original purpose was to add information on support for CREATE TRANSFORM (which must be explicitly coded in any PL implementation intending to support it). But the plhandler section was about as long as a monolith of text ought to be, even before adding transform information, so reorganized first into sections. Front-loaded with short descriptions of the three possible functions (call handler, validator, inline handler) registered with CREATE LANGUAGE. The latter two were afterthoughts in historical sequence, but the docs don't need to present them that way. The section had also fallen behind the introduction of procedures, so updated to generally use the umbrella term 'routine' in place of 'function'. New section markup added here without indentation change, to avoid obscuring changes. A follow-on commit will reindent and rewrap. --- doc/src/sgml/event-trigger.sgml | 16 ++ doc/src/sgml/plhandler.sgml | 400 ++++++++++++++++++++++++++------- doc/src/sgml/ref/create_transform.sgml | 65 ++++-- doc/src/sgml/xfunc.sgml | 2 +- 4 files changed, 393 insertions(+), 90 deletions(-) diff --git a/doc/src/sgml/event-trigger.sgml b/doc/src/sgml/event-trigger.sgml index 9c66f97..14ec70a 100644 --- a/doc/src/sgml/event-trigger.sgml +++ b/doc/src/sgml/event-trigger.sgml @@ -670,6 +670,14 @@ + CREATE TRANSFORM + X + X + - + - + + + CREATE TRIGGER X X @@ -958,6 +966,14 @@ + DROP TRANSFORM + X + X + X + - + + + DROP TRIGGER X X diff --git a/doc/src/sgml/plhandler.sgml b/doc/src/sgml/plhandler.sgml index 40ee59d..95e8515 100644 --- a/doc/src/sgml/plhandler.sgml +++ b/doc/src/sgml/plhandler.sgml @@ -1,7 +1,7 @@ - Writing a Procedural Language Handler + Implementing a new Procedural Language procedural language @@ -9,52 +9,117 @@ - All calls to functions that are written in a language other than - the current version 1 interface for compiled - languages (this includes functions in user-defined procedural languages - and functions written in SQL) go through a call handler - function for the specific language. It is the responsibility of - the call handler to execute the function in a meaningful way, such - as by interpreting the supplied source text. This chapter outlines - how a new procedural language's call handler can be written. + To make a new procedural language available in + PostgreSQL, at least one dedicated handler + function, and optionally one or two others, must be written and then named + in a command. They are: + + + call handler + + + (Required.) Responsible for executing + routines + (functions or + procedures) defined + in the procedural language. + + + + + validator + + + (Optional.) If provided, this will be called whenever a routine using + the procedural language has been created or updated, and should check + the definition to report any errors detectable at that time. + + + + + inline handler + + + (Optional.) If the procedural language can be used inline in + a statement, this handler must be provided, + and is responsible for executing such inline code. + + + + - The call handler for a procedural language is a - normal function that must be written in a compiled - language such as C, using the version-1 interface, and registered - with PostgreSQL as taking no arguments + This chapter outlines how a new procedural language's handlers can be + written. + + + + Call handler function + + + Every routine defined with a language name other than + internal (as defined in ) + or c () will be called by + invoking the procedural language's call handler. + + + This is true even of routines with language name sql, + though, as a special case, that call handler has no entry in the system + catalogs. + + + It is the responsibility of + the call handler to execute the routine in a meaningful way, such + as by interpreting the supplied source text. + + + + The call handler is a normal user-defined function that must + be declared to PostgreSQL as taking no arguments and returning the type language_handler. This special pseudo-type identifies the function as a call handler and prevents it from being called directly in SQL commands. - For more details on C language calling conventions and dynamic loading, - see . + This handler must not require a call handler of its own, which makes + the predefined languages internal or c + the only choices for the handler's own declaration. + Typically, it will be a loadable function in language + c, as described in . + + + It may be implemented in a language other than C, as long as it can be + built into a loadable object with compatible calling conventions. + + The call handler is called in the same way as any other function: It receives a pointer to a FunctionCallInfoBaseData struct containing - argument values and information about the called function, and it + argument values and information about the called routine, and it is expected to return a Datum result (and possibly set the isnull field of the FunctionCallInfoBaseData structure, if it wishes to return an SQL null result). The difference between a call - handler and an ordinary callee function is that the + handler and an ordinary callee is that the flinfo->fn_oid field of the FunctionCallInfoBaseData structure will contain - the OID of the actual function to be called, not of the call + the OID of the actual routine to be called, not of the call handler itself. The call handler must use this field to determine - which function to execute. Also, the passed argument list has - been set up according to the declaration of the target function, + which routine to execute. Also, the passed argument list has + been set up according to the declaration of the target routine, not of the call handler. - It's up to the call handler to fetch the entry of the function from the - pg_proc system catalog and to analyze the argument - and return types of the called function. The AS clause from the - CREATE FUNCTION command for the function will be found + It's up to the call handler to fetch the routine's defining + pg_proc row from the system catalog cache + to determine what to execute, what parameter and return types are expected, + and so on. + The AS clause from the + CREATE FUNCTION or CREATE PROCEDURE + command for the routine will be found in the prosrc column of the pg_proc row. This is commonly source text in the procedural language, but in theory it could be something else, @@ -63,12 +128,147 @@ - Often, the same function is called many times per SQL statement. + The handler may also examine the passed + FunctionCallInfoBaseData structure for information + on the context of the call. If the procedural language will support + returning sets, the structure may contain a pointer to a + ReturnSetInfo structure for use as described in + . If the language will support triggers + or event triggers, the structure may hold a pointer to one of the structures + described in or + , and the procedural language + should provide some way for the called function to use the information + they carry. + + + + Parameter and return type resolution + + + A routine's statically-declared parameter types (and, for a function, + return type) are found in the proargtypes and + prorettype columns of the pg_proc + row. + If a routine has OUT parameters, those types are + included in the proallargtypes column, and their names + in proargnames. + Convenience functions declared in funcapi.h are + available for extracting that information. + + + + The statically-declared types may include polymorphic types that need + to be resolved according to the actual types present at the call site, + as described in . + + + + + + Mapping to procedural language types + + + Once the PostgreSQL types of any parameters + and results have been resolved, the handler must determine how it will + map their values to and from suitable types that exist in the procedural + language. + + + + The designer of a procedural language will typically document what types + will be supported and how they will be mapped, which could be as simple + as using every type's text input/output format to map it to the target + language's string type, or could directly map many types to corresponding + ones the target language provides. The handler function will implement + those rules. + + + + Type transforms + + + Because PostgreSQL is extensible, and + an extension can easily supply new types, a procedural language handler + may encounter types it has no predefined mappings for, or only an awkward + default mapping such as to a text string. A procedural language can be + designed so that its type mappings are also extensible, and an extension + can add mappings between new PostgreSQL types + and suitable types in the target language. + + + + One mechanism PostgreSQL provides that may be + used for that purpose is . + The command associates a PostgreSQL type and + a specific procedural language with a pair of functions to handle the + mapping of that type to a corresponding procedural language type and back. + + + + For a procedural language to support transforms, its call handler is + responsible for consulting the protrftypes column of + a routine's pg_proc row to determine which types + should have transforms applied. + A convenience function get_call_trftypes is + available. + The call handler must then resolve the from SQL function + for each affected parameter type, and the to SQL function + for any affected result. + It may use the get_transform_fromsql and + get_transform_tosql functions for that. + + + + The handler must then apply the proper from SQL functions + to all affected inputs (including elements within array or composite + types) and, after calling the target routine, apply the proper + to SQL functions similarly to any results. + If the target routine might interact with the database using SPI, + the handler may arrange for the requested transforms to be applied + in those operations as well. + + + + Because the procedural language implementation, and not + PostgreSQL itself, is responsible for calling + the transform functions, it is free to define what it will pass as the + parameter to each function (declared as internal for both), + and how it will interpret the result (also declared internal) + of the from SQL function. Effectively, each procedural + language's implementation defines the API that must be adhered to + by any author of transforms for that language. + + + + A procedural language might impose limits on where and how it will apply + transforms (such as on array or domain types). The + get_transform_fromsql and + get_transform_tosql functions mentioned above + consider each type only shallowly, and will not, for example, return + a transform function for a domain type if only its base type was listed in + the TRANSFORM clause. + If a procedural language's call handler does not implement transforms + at all, no TRANSFORM clause will have any effect + for routines declared in that language. + The language's validator function can be used to give immediate feedback + if a routine is declared with TRANSFORM clauses + the implementation cannot support. + + + + + + + + Caching resolved routine information + + + Often, the same routine is called many times per SQL statement. A call handler can avoid repeated lookups of information about the - called function by using the + called routine by using the flinfo->fn_extra field. This will initially be NULL, but can be set by the call handler to point at - information about the called function. On subsequent calls, if + information about the called routine. On subsequent calls, if flinfo->fn_extra is already non-NULL then it can be used and the information lookup step skipped. The call handler must make sure that @@ -81,59 +281,58 @@ normally have the same lifespan as the FmgrInfo itself. But the handler could also choose to use a longer-lived memory context so that it can cache - function definition information across queries. + routine definition information across queries. - When a procedural-language function is invoked as a trigger, no arguments - are passed in the usual way, but the - FunctionCallInfoBaseData's - context field points at a - TriggerData structure, rather than being NULL - as it is in a plain function call. A language handler should - provide mechanisms for procedural-language functions to get at the trigger - information. + If the handler supports returning sets, and uses the ValuePerCall mode + helper macros described in , it must + not use fn_extra during set-returning calls. + The helper macros use that field for their own purposes. After + SRF_FIRSTCALL_INIT has been called, the field will point + to a FuncCallContext structure, which has + a user_fctx field that can be used similarly, + but only through the sequence of calls returning one set result. - - A template for a procedural-language handler written as a C extension is - provided in src/test/modules/plsample. This is a - working sample demonstrating one way to create a procedural-language - handler, process parameters, and return a value. - + - - Although providing a call handler is sufficient to create a minimal - procedural language, there are two other functions that can optionally - be provided to make the language more convenient to use. These - are a validator and an - inline handler. A validator can be provided - to allow language-specific checking to be done during - . - An inline handler can be provided to allow the language to support - anonymous code blocks executed via the command. - + + + + Validator function If a validator is provided by a procedural language, it must be declared as a function taking a single parameter of type oid. The validator's result is ignored, so it is customarily - declared to return void. The validator will be called at - the end of a CREATE FUNCTION command that has created - or updated a function written in the procedural language. - The passed-in OID is the OID of the function's pg_proc + declared to return void. + The validator itself may be written in any procedural language able to + receive an oid-typed parameter and query system catalogs. + + + + The validator will be called at + the end of a CREATE FUNCTION or + CREATE PROCEDURE command that has created + or updated a routine written in the procedural language. + The passed-in OID is the OID of the routine's pg_proc row. The validator must fetch this row in the usual way, and do whatever checking is appropriate. + + + First, call CheckFunctionValidatorAccess() to diagnose explicit calls to the validator that the user could not achieve through - CREATE FUNCTION. Typical checks then include verifying - that the function's argument and result types are supported by the - language, and that the function's body is syntactically correct - in the language. If the validator finds the function to be okay, + CREATE FUNCTION or CREATE PROCEDURE. + Typical checks then include verifying + that the routine's argument and result types are supported by the + language, and that the routine's body is syntactically correct + in the language. If the validator finds the routine to be okay, it should just return. If it finds an error, it should report that via the normal ereport() error reporting mechanism. Throwing an error will force a transaction rollback and thus prevent - the incorrect function definition from being committed. + the incorrect routine definition from being committed. @@ -143,35 +342,72 @@ language provides for code execution at compilation time, the validator must suppress checks that would induce such execution. In particular, this parameter is turned off by pg_dump so that it can - load procedural language functions without worrying about side effects or - dependencies of the function bodies on other database objects. + load procedural language routines without worrying about side effects or + dependencies of the routine bodies on other database objects. (Because of this requirement, the call handler should avoid - assuming that the validator has fully checked the function. The point + assuming that the validator has fully checked the routine. The point of having a validator is not to let the call handler omit checks, but to notify the user immediately if there are obvious errors in a - CREATE FUNCTION command.) + CREATE FUNCTION or CREATE PROCEDURE + command.) + + + While the choice of exactly what to check is mostly left to the discretion of the validator function, note that the core - CREATE FUNCTION code only executes SET clauses - attached to a function when check_function_bodies is on. + CREATE FUNCTION and CREATE PROCEDURE + code only executes SET clauses + attached to a routine when check_function_bodies is on. Therefore, checks whose results might be affected by GUC parameters definitely should be skipped when check_function_bodies is off, to avoid false failures when reloading a dump. - If an inline handler is provided by a procedural language, it + If a language's call handler does not apply parameter and return type + transforms, then no TRANSFORM clause in a routine + declaration will have any effect. To provide immediate feedback if a + declaration contains such a clause, the validator can report a suitable + error whenever the protrftypes column of the routine's + pg_proc row is non-null. + + + + + + Inline handler function + + + If this handler is provided by a procedural language, it must be declared as a function taking a single parameter of type - internal. The inline handler's result is ignored, so it is - customarily declared to return void. The inline handler + internal, which will be a pointer + to an InlineCodeBlock struct when the handler + is called. The result is ignored, so the return type is customarily + declared as void. + The inline handler itself may be written in any procedural language that + permits declaring an internal parameter with a suitable + language binding for accessing it as an + InlineCodeBlock struct. + + + + The inline handler will be called when a DO statement is executed specifying - the procedural language. The parameter actually passed is a pointer - to an InlineCodeBlock struct, which contains information + the procedural language. The InlineCodeBlock + struct contains information about the DO statement's parameters, in particular the - text of the anonymous code block to be executed. The inline handler - should execute this code and return. + text of the anonymous code block to be executed. + It also contains the OID of the intended procedural language and whether + that procedural language is declared as TRUSTED, useful + if a single inline handler is supporting more than one procedural language. + The inline handler should execute the code block and return. + + + + Packaging the language handlers + It's recommended that you wrap all these function declarations, as well as the CREATE LANGUAGE command itself, into @@ -181,6 +417,18 @@ extensions. + + + + Example code + + + A template for a procedural-language handler written as a C extension is + provided in src/test/modules/plsample. This is a + working sample demonstrating one way to create a procedural-language + handler, process parameters, and return a value. + + The procedural languages included in the standard distribution are good references when trying to write your own language handler. @@ -189,4 +437,6 @@ reference page also has some useful details. + + diff --git a/doc/src/sgml/ref/create_transform.sgml b/doc/src/sgml/ref/create_transform.sgml index 34bdc60..e1dc79a 100644 --- a/doc/src/sgml/ref/create_transform.sgml +++ b/doc/src/sgml/ref/create_transform.sgml @@ -22,8 +22,8 @@ PostgreSQL documentation CREATE [ OR REPLACE ] TRANSFORM FOR type_name LANGUAGE lang_name ( - FROM SQL WITH FUNCTION from_sql_function_name [ (argument_type [, ...]) ], - TO SQL WITH FUNCTION to_sql_function_name [ (argument_type [, ...]) ] + FROM SQL WITH FUNCTION from_sql_function_name [ (internal) ], + TO SQL WITH FUNCTION to_sql_function_name [ (internal) ] ); @@ -80,6 +80,19 @@ CREATE [ OR REPLACE ] TRANSFORM FOR type_name LANGUAG have EXECUTE privilege on the from-SQL and to-SQL functions, if specified. + + + Typically, CREATE TRANSFORM will be used to register + transforms developed by others (or, even more typically, they will be + packaged in an extension, and CREATE EXTENSION will + do the registration). + To develop new transform functions for a given procedural language, you must + be familiar with its implementation, which determines whether it supports + transforms at all, and defines the details, such as what is expected for + the return and parameter types declared internal. + To develop a procedural language with support for transforms, see + . + @@ -107,18 +120,19 @@ CREATE [ OR REPLACE ] TRANSFORM FOR type_name LANGUAG - from_sql_function_name[(argument_type [, ...])] + from_sql_function_name[(internal)] The name of the function for converting the type from the SQL environment to the language. It must take one argument of type internal and return type internal. The - actual argument will be of the type for the transform, and the function - should be coded as if it were. (But it is not allowed to declare an - SQL-level function returning internal without at - least one argument of type internal.) The actual return - value will be something specific to the language implementation. + actual argument will be supplied by the language handler. It will + typically be of the type for the transform (even though declared + internal), but when writing transforms for a particular + language, its implementation should be consulted for the details. + The actual return + value will also be something specific to the language implementation. If no argument list is specified, the function name must be unique in its schema. @@ -126,7 +140,7 @@ CREATE [ OR REPLACE ] TRANSFORM FOR type_name LANGUAG - to_sql_function_name[(argument_type [, ...])] + to_sql_function_name[(internal)] @@ -192,11 +206,34 @@ CREATE TRANSFORM FOR hstore LANGUAGE plpython3u ( Compatibility - This form of CREATE TRANSFORM is a - PostgreSQL extension. There is a CREATE - TRANSFORM command in the SQL standard, but it - is for adapting data types to client languages. That usage is not supported - by PostgreSQL. + There is a CREATE TRANSFORM command in the + SQL standard. + LANGUAGE lang_name is a + PostgreSQL extension. + A transform in the SQL standard may be created only for + a user-defined type, and transforms it to a standard + SQL type that is expected to have cross-language support. + A PostgreSQL transform can be created for + any type, and transforms it directly to a language-specific type. + + + + A transform in the SQL standard is applied + for server-side routines and also for embedded-SQL client + languages. In PostgreSQL, a transform is only + applied for server-side routines. + + + + In the SQL standard, a transform group (one pair of + from-SQL and to-SQL functions) may have + a group name, allowing multiple transforms to exist for the same type, and + each routine may declare which one to apply, by giving its name. + Transform groups are not named in PostgreSQL, + so only one transform for a given type and procedural language may exist at + any time. Routines using that language may choose between two mappings for + the type: the built-in default, or the one defined by the single available + transform. diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index a347230..df613e6 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -3295,7 +3295,7 @@ CREATE OR REPLACE FUNCTION retcomposite(IN integer, IN integer, - + Polymorphic Arguments and Return Types -- 2.7.3