Re: Parser Hook

On Mon, Mar 15, 2021 at 12:43 PM Julien Rouhaud <rjuju123(at)gmail(dot)com> wrote:

> On Mon, Mar 15, 2021 at 11:48:58AM -0400, Jim Mlodgenski wrote:
> >
> > Going deeper on this, I created another POC as an example. Yes, having a
> > hook at the top of the parser does mean an extension needs to copy the
> > existing grammar and modify it. Without a total redesign of how the
> grammar
> > is handled, I'm not seeing how else this could be accomplished. The
> example
> > I have is adding a CREATE JOB command that a scheduler may use. The
> amount
> > of effort needed for an extension maintainer doesn't appear to be that
> > onerous. Its not ideal having to copy and patch gram.y, but certainly
> > doable for someone wanting to extend the parser.
>
> AFAIK nothing in bison prevents you from silently ignoring unhandled
> grammar
> rather than erroring out. So you could have a parser hook called first,
> and
> if no valid command was recognized fall back on the original parser. I'm
> not
> saying that it's a good idea or will be performant (although the added
> grammar
> will likely be very small, so it may not be that bad), but you could
> definitely
> avoid the need to duplicate the whole grammar in each and every extension,
> and
> allow multiple extensions extending the grammar.
>
>
That's a good point. That does simplify it

> That won't reduce the difficulty of producing a correct parse tree if you
> want
> to implement some syntactic sugar for already handled DML though.
>

Complex DML like Oracle's outer join syntax is tricky no matter which way
you slice it.

> > However we would want to modify the parser to allow it to be more
> plugable
> > in the future, we would very likely need to have a hook at the top of the
> > parser to intiailize things like keywords. Having a hook at the top of
> the
> > parser along with the existing ProcessUtility_hook allows extension to
> add
> > their own utility commands if they wish. I would image that covers many
> > existing use cases for extensions today.
>
> What happens if multiple extensions want to add their own new grammar?
> There
> will at least be possible conflicts with the additional node tags.
>

The extensions would need to play nice with one another like they do with
other hooks and properly call the previous hook.

> Also, I'm not sure that many extensions would really benefit from custom
> utility command, as you can already do pretty much anything you want using
> SQL
> functions. For instance it would be nice for hypopg to be able to support
>
> CREATE HYPOTHETICAL INDEX ...
>
> rather than
>
> SELECT hypopg_create_index('CREATE INDEX...')
>
> But really the only benefit would be autocompletion, which still wouldn't
> be
> possible as psql autocompletion won't be extended. And even if it somehow
> was,
> I wouldn't expect all psql clients to be setup as needed.
>

Having the functionality exposed through DDL gives it a more native feel to
it for users and for some more likely use the exentions.