Re: Making cancellations safe

On Wed, 4 Nov 2020 at 10:50, Shay Rojansky <roji(at)roji(dot)org> wrote:

> Hi all.
>
> Back in 2016 I started a thread about making cancellations safer[1], I'd
> like to try to pick this up again. Here a summary of the previous
> conversation:
>
> The main ask here is to allow clients to specify which command to cancel,
> to avoid various race conditions where the wrong command is accidentally
> cancelled. The implementation proposal by Tom was for the client and server
> to simply count messages on each side, and to modify the CancelRequest
> message for the client to integrate the requested sequence number to be
> cancelled.
>
> The CancelRequest format change can probably happen in a non-breaking way,
> e.g. by defining a new cancel request code (the 1234/5678 at the beginning
> of the message). The client could know that the server is able to accept
> the new CancelRequest format via a minor protocol bump, or some other means.
>
> There was also discussion of allowing "cancel up to" semantics, where all
> messages up to the provided sequence number are cancelled. While this could
> be useful, it's important to allow setting a lower bound as well. A new
> experimental mode in the Npgsql driver "multiplexes" queries from unrelated
> threads into the same physical connection, possibly pipelining two
> unrelated queries go into a single outgoing TCP packet. In this case, the
> second query producer may issue a cancellation, and unintentionally cancel
> the first query which is still in progress. So ideally the CancelRequest
> would contain both a lower and an upper bound (with the lower being
> optional).
>
> Since cancellation requests may arrive before their target queries do, the
> backend should be able to track unfulfilled requests and apply them as soon
> as the target query is received.
>
> Finally, there was also an orthogonal discussion about widening the cancel
> key (currently 32-bit). However, that seems like it would be a protocol
> breaking change, so possibly better treated separately.
>
> [1]
> https://www.postgresql.org/message-id/flat/CADT4RqDh1CEgz7QgKwYSLT9TMCk7O%3DncauUaSQKVt_nPNTE9wQ%40mail.gmail.com#00832c72f4b93d57d6b0ac59de8eca85
>

I think this is going to become more relevant as libraries push the
protocol. Currently we have a number of implementations of pipelining where
this would be useful.

In general I think we need to have a way to cancel a specific query.

Regards,

Dave