Re: [HACKERS] Pipelining executions to postgresql server

From: Mikko Tiihonen <Mikko(dot)Tiihonen(at)nitorcreations(dot)com>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, "pgsql-jdbc(at)postgresql(dot)org" <pgsql-jdbc(at)postgresql(dot)org>
Subject: Re: [HACKERS] Pipelining executions to postgresql server
Date: 2014-11-02 12:11:49
Message-ID: 1414930308808.49160@nitorcreations.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-jdbc

> From: Andres Freund <andres(at)2ndquadrant(dot)com>
> On 2014-11-01 14:04:05 +0000, Mikko Tiihonen wrote:
> > I created a proof of concecpt patch for postgresql JDBC driver that
> > allows the caller to do pipelining of requests within a
> > transaction. The pipelining here means same as for HTTP: the client
> > can send the next execution already before waiting for the response of
> > the previous request to be fully processed.
>
> Slightly confused here. To my knowledge the jdbc driver already employs
> some pipelining? There's some conditions where it's disabled (IIRC
> RETURNING for DML is one of them), but otherwise it's available.
>
> I'm very far from a pgjdbc expert, but that's what I gathered from the
> code when investigating issues a while back and from my colleague Craig.

Most DB interfaces make the server operations look synchronous.
For JDBC the only standard interface is similar to libpg:
PGresult *PQexec(PGconn *conn, const char *command);

I should have searched earlier a better reference to libpg. I am planning on adding support for something similar to
http://www.postgresql.org/docs/9.3/static/libpq-async.html
More specifically operations like:
int PQsendQuery(PGconn *conn, const char *command);
PGresult *PQgetResult(PGconn *conn);
The Java API will of course be custom to postgresql jdbc driver since there is no official java api for async db operations.

The above I can do purely on the jdbc driver side. But my my question was about pipelining.
In libpg terms: Is it safe to do multiple PQsendQuery operations before invoking PQgetResult as many times?
It should be, if the server startd to process (read in) the next query only after it has sent the previous response out.

And do any pg connection poolers support having multiple executions on-the-fly at the same time for the same connection?

-Mikko

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message rohtodeveloper 2014-11-02 12:22:25 Re: How to implent CONVERT ( data_type [ ( length ) ] , expression ) function in postgreSQL
Previous Message Simon Riggs 2014-11-02 10:39:22 Re: group locking: incomplete patch, just for discussion

Browse pgsql-jdbc by date

  From Date Subject
Next Message Andres Freund 2014-11-02 12:23:20 Re: [HACKERS] Pipelining executions to postgresql server
Previous Message Andres Freund 2014-11-01 17:31:16 Re: Pipelining executions to postgresql server