Re: [HACKERS] Pipelining executions to postgresql server

> > > From: Andres Freund <andres(at)2ndquadrant(dot)com>
> > > On 2014-11-01 14:04:05 +0000, Mikko Tiihonen wrote:
> > > > I created a proof of concecpt patch for postgresql JDBC driver that
> > > > allows the caller to do pipelining of requests within a
> > > > transaction. The pipelining here means same as for HTTP: the client
> > > > can send the next execution already before waiting for the response of
> > > > the previous request to be fully processed.
> > >
> > > Slightly confused here. To my knowledge the jdbc driver already employs
> > > some pipelining? There's some conditions where it's disabled (IIRC
> > > RETURNING for DML is one of them), but otherwise it's available.
> > >
> > > I'm very far from a pgjdbc expert, but that's what I gathered from the
> > > code when investigating issues a while back and from my colleague Craig.
> >
> > Most DB interfaces make the server operations look synchronous.
>
> You IIRC can use jdbc's batch interface.

Yes, there is a limited batch interface for inserts and updates. But for example when using prepared statements you can only do batches of same statement (with different parameters of course).

> > I should have searched earlier a better reference to libpg. I am planning on adding support for something similar to
> > http://www.postgresql.org/docs/9.3/static/libpq-async.html
> > More specifically operations like:
> > int PQsendQuery(PGconn *conn, const char *command);
> > PGresult *PQgetResult(PGconn *conn);
>
> The network protocol allows for pipelining, yes. But libpq unfortunately
> doesn't.
> You should read the protocol definition.

I have studied the protocol, that is why I concluded that it would be possible to add support for pipelining for clients.

> > It should be, if the server startd to process (read in) the next query
> > only after it has sent the previous response out.
>
> There's complexities around error handling and such making it slightly
> more complex.

Are you referring to some complexities on the server side related to error handling or on the client side?

Is the following summary correct:
- the network protocol supports pipelinings
- the server handles operations in order, starting the processing of next operation only after fully processing the previous one - thus pipelining is invisible to the server
- libpq driver does not support pipelining, but that is due to internal limitations
- if proper error handling is done by the client then there is no reason why pipelining could be supported by any pg client

-Mikko