libpq Alternate Row Processor

From: Kyle Gearhart <kyle(dot)gearhart(at)indigohill(dot)io>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: libpq Alternate Row Processor
Date: 2017-02-03 21:53:36
Message-ID: BLUPR14MB0162DA841005B027F4CA1465FA4F0@BLUPR14MB0162.namprd14.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

The guts of pqRowProcessor in libpq does a good bit of work to maintain the internal data structure of a PGresult. There are a few use cases where the caller doesn't need the ability to access the result set row by row, column by column using PQgetvalue. Think of an ORM that is just going to copy the data from PGresult for each row into its own structures.

I've got a working proof of concept that allows the caller to attach a callback that pqRowProcessor will call instead of going thru its own routine. This eliminates all the copying of data from the PGconn buffer to a PGresult buffer and then ultimately a series of PQgetvalue calls by the client. The callback allows the caller to receive each row's data directly from the PGconn buffer.

It would require exposing struct pgDataValue in libpq-fe.h. The prototype for the callback pointer would be:
int (*PQrowProcessorCB)(PGresult*, const PGdataValue*, int col_count, void *user_data);

My initial testing shows a significant performance improvement. I'd like some opinions on this before wiring up a performance proof and updating the documentation for a formal patch submission.

Kyle Gearhart

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2017-02-03 21:55:21 Re: pageinspect: Hash index support
Previous Message Robert Haas 2017-02-03 21:39:08 Re: [PATCH] Rename pg_switch_xlog to pg_switch_wal