Re: libpq Alternate Row Processor

On 2/13/17 8:46 AM, Kyle Gearhart wrote:
> profile_filler.txt
> 61,410,901 ???:_int_malloc [/usr/lib64/libc-2.17.so]
> 38,321,887 ???:_int_free [/usr/lib64/libc-2.17.so]
> 31,400,139 ???:pqResultAlloc [/usr/local/pgsql/lib/libpq.so.5.10]
> 22,839,505 ???:pqParseInput3 [/usr/local/pgsql/lib/libpq.so.5.10]
> 17,600,004 ???:pqRowProcessor [/usr/local/pgsql/lib/libpq.so.5.10]
> 16,002,817 ???:malloc [/usr/lib64/libc-2.17.so]
> 14,716,359 ???:pqGetInt [/usr/local/pgsql/lib/libpq.so.5.10]
> 14,400,000 ???:check_tuple_field_number [/usr/local/pgsql/lib/libpq.so.5.10]
> 13,800,324 main.c:main [/usr/local/src/postgresql-perf/test]

> profile_filler_callback.txt
> 16,842,303 ???:pqParseInput3 [/usr/local/pgsql/lib/libpq.so.5.10]
> 14,810,783 ???:_int_malloc [/usr/lib64/libc-2.17.so]
> 12,616,338 ???:pqGetInt [/usr/local/pgsql/lib/libpq.so.5.10]
> 10,000,000 ???:pqSkipnchar [/usr/local/pgsql/lib/libpq.so.5.10]
> 9,200,004 main.c:process_callback [/usr/local/src/postgresql-perf/test]

Wow, that's a heck of a difference.

There's a ton of places where the backend copies data for no other
purpose than to put it into a different memory context. I'm wondering if
there's improvement to be had there as well, or whether palloc is so
much faster than malloc that it's not an issue. I suspect that some of
the effects are being masked by other things since presumably palloc and
memcpy are pretty cheap on small volumes of data...
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)