pgsql: Re-refactor the core scanner's API, in order to get out from

From: tgl(at)postgresql(dot)org (Tom Lane)
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Re-refactor the core scanner's API, in order to get out from
Date: 2009-11-09 18:38:49
Message-ID: 20091109183849.ADBCC753FB7@cvs.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Log Message:
-----------
Re-refactor the core scanner's API, in order to get out from under the problem
of different parsers having different YYSTYPE unions that they want to use
with it. I defined a new union core_YYSTYPE that is just the (very short)
list of semantic values returned by the core scanner. I had originally
worried that this would require an extra interface layer, but actually we can
have parser.c's base_yylex (formerly filtered_base_yylex) take care of that at
no extra cost. Names associated with the core scanner are now "core_yy_foo",
with "base_yy_foo" being used in the core Bison parser and the parser.c
interface layer.

This solves the last serious stumbling block to eliminating plpgsql's separate
lexer. One restriction that will still be present is that plpgsql and the
core will have to agree on the token numbers assigned to tokens that can be
returned by the core lexer. Since Bison doesn't seem willing to accept
external assignments of those numbers, we'll have to live with decreeing that
core and plpgsql grammars declare these tokens first and in the same order.

Modified Files:
--------------
pgsql/src/backend/parser:
gram.y (r2.689 -> r2.690)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/parser/gram.y?r1=2.689&r2=2.690)
parser.c (r1.81 -> r1.82)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/parser/parser.c?r1=1.81&r2=1.82)
scan.l (r1.162 -> r1.163)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/parser/scan.l?r1=1.162&r2=1.163)
pgsql/src/include/parser:
gramparse.h (r1.49 -> r1.50)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/parser/gramparse.h?r1=1.49&r2=1.50)

Added Files:
-----------
pgsql/src/include/parser:
scanner.h (r1.1)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/parser/scanner.h?rev=1.1&content-type=text/x-cvsweb-markup)

Browse pgsql-committers by date

  From Date Subject
Next Message Robert Haas 2009-11-09 18:42:47 Re: Re: [COMMITTERS] pgsql: Rewrite GEQO's gimme_tree function so that it always finds a
Previous Message Tom Lane 2009-11-09 18:10:11 Re: Re: [COMMITTERS] pgsql: Rewrite GEQO's gimme_tree function so that it always finds a