Re: Improved scanner performance

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> I've used 'flex -8 -CFa' and restructured the code to avoid looping over
> and copying the input string half a dozen times. For instance, instead of
> scanstr(), the escape sequences are resolved as the input is scanned, and
> instead of the myinput() routine I use the function yy_scan_buffer()
> provided by flex for scanning in-memory strings. (This would make the
> code flex-dependent, but in reality it already is anyway.)

Yes, we've been requiring flex-only features for years, so that aspect
of it doesn't bother me. Any downsides to the changes? (For instance,
I had the idea that -CF would enlarge the lexer tables quite a bit ---
what's the change in executable size?)

> The "after" profile is:

> % cumulative self self total
> time seconds seconds calls ms/call ms/call name
> 40.30 5.65 5.65 11 513.64 943.64 pq_getstring
> 33.74 10.38 4.73 74882482 0.00 0.00 pq_getbyte
> 18.90 13.03 2.65 110 24.09 24.09 base_yylex
> 6.85 13.99 0.96 22 43.64 43.64 textin
> 0.07 14.00 0.01 86 0.12 0.12 heap_fetch

Looks like inlining pq_getbyte into pq_getstring would be worth doing
too.

regards, tom lane