Re: Improved scanner performance

BTW, here is what I get on an HP box for the same test you described
(a dozen trivial SELECTs using string literals between 5MB and 10MB
long), using latest sources:

% cumulative self self total
time seconds seconds calls ms/call ms/call name
47.51 8.19 8.19 chunks
26.16 12.70 4.51 129 34.96 35.97 base_yylex
12.30 14.82 2.12 1521 1.39 1.39 strlen
6.79 15.99 1.17 13 90.00 90.00 pq_getstring
4.18 16.71 0.72 chunk2
2.55 17.15 0.44 _recv_sys
0.29 17.20 0.05 _mcount

"chunks" is the inner loop of memcpy() --- evidently all the time is
being spent just copying those big literals around.

We could probably avoid some of that copying if we had a cleaner
approach to parsetree handling, ie, no scribbling on one's input.
Then operations like eval_const_expressions wouldn't feel compelled
to copy parsetree nodes that they weren't modifying. But I think
we've gotten all the low-hanging fruit for now.

At least on the backend side. Did you notice that psql was chewing
up three times more CPU than the backend in this test??

regards, tom lane