Re: Pipelining executions to postgresql server

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Mikko Tiihonen <Mikko(dot)Tiihonen(at)nitorcreations(dot)com>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, "pgsql-jdbc(at)postgresql(dot)org" <pgsql-jdbc(at)postgresql(dot)org>
Subject: Re: Pipelining executions to postgresql server
Date: 2014-11-01 16:13:03
Message-ID: 4863.1414858383@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-jdbc

Mikko Tiihonen <Mikko(dot)Tiihonen(at)nitorcreations(dot)com> writes:
> I created a proof of concecpt patch for postgresql JDBC driver that allows the caller to do pipelining of requests within a transaction. The pipelining here means same as for HTTP: the client can send the next execution already before waiting for the response of the previous request to be fully processed.

In principle this can work if you think through the error handling
carefully. The way the FE/BE protocol is intended to handle the case
is that all the pipelined statements are part of one transaction so
that they all succeed or fail together. Otherwise the user has to work
through and predict the outcome of all the combinations of "Q1 succeeded,
Q2 failed, Q3 succeeded" which is mighty error-prone, especially if the
commands are interdependent in any way at all. Having legislated that
(and forbidden any transaction-boundary commands in a pipelined group),
what you do is not issue a Sync until after the last command of the set.
Then, if any command fails, the backend just automatically discards
remaining messages until it gets the Sync, and so you can safely issue
new commands before knowing the results of the previous ones.

I mention this because, although I don't know the innards of the JDBC
driver at all, it sure doesn't look like that's the way you've done it.
If I'm right that you've left the error recovery strategy to the user,
I think this is gonna be very hard to use safely.

regards, tom lane

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2014-11-01 16:57:40 Re: _mdfd_getseg can be expensive
Previous Message Marko Tiikkaja 2014-11-01 14:52:50 Re: pgcrypto: PGP signatures

Browse pgsql-jdbc by date

  From Date Subject
Next Message Andres Freund 2014-11-01 17:31:16 Re: Pipelining executions to postgresql server
Previous Message Mikko Tiihonen 2014-11-01 14:04:05 Pipelining executions to postgresql server