Re: Upgrading our minimum required flex version for 8.5

2009/7/12 Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> Tom Lane wrote:
>>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>>> I think it would need to be benchmarked. My faint recollection is that
>>>> the re-entrant lexers are slower.
>>>
>>> The flex documentation states in so many words:
>>>    The option `--reentrant' does not affect the performance of the scanner.
>>> Do you feel a need to verify their claim?
>
>> No, I'll take their word for it. I must have been thinking of something
>> else.
>
> As I got further into this, it turned out that Andrew's instinct was
> right: it does need to be benchmarked.  Although the inner loops of the
> lexer seem to be the same with or without --reentrant, once you buy into
> the whole nine yards of --reentrant, --bison-bridge, and a "pure" bison
> parser, you find out that the lexer's API changes: there are more
> parameters to yylex() than there used to be.  It's also necessary to
> pass around a yyscanner pointer to all the subroutines in scan.l.  (But
> on the other hand, this eliminates accesses to global variables, which
> are often not that cheap.)  So the "no performance impact" claim isn't
> telling the whole truth.
>
> As best I can tell after some casual testing on a couple of machines,
> the actual bottom line is that "raw_parser" (ie, the bison and flex
> processing) is going to be a couple of percent slower with a reentrant
> grammar and lexer, for typical queries involving a lot of short tokens.
> Now this disappears into the noise as soon as you include parse analysis
> (let alone planning and execution), but it is possible to measure the
> slowdown in a test harness that calls raw_parser only.
>
> A possible compromise that I think would avoid most or all of the
> slowdown is to make the lexer reentrant but not the grammar (so that
> yylval and yylloc remain as global variables instead of being parameters
> to yylex).  I haven't actually benchmarked that, though.  It strikes
> me as a fairly silly thing to do.  If we're going to go for reentrancy
> I think we should fix both components.

when we don't use reentrant grammar, then we cannot use main sql parser in SQL?

Pavel

>
> I'm willing to live with the small slowdown.  Comments?
>
>                        regards, tom lane
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>