Re: Proposal: http2 wire format

On 28 March 2018 at 16:02, Andres Freund <andres(at)anarazel(dot)de> wrote:

> On 2018-03-26 22:44:09 +0200, Damir Simunic wrote:
> > > *NONE* of the interesting problems are solved by HTTP2. You *still*
> > > need a full blown protocol ontop of it. So no, this doesn't change
> that.
> >
> > If you had to nominate only one of those problems, which one would you
> consider the most interesting?
>
> A few random, very tired, points:
>
> - consolidated message for common tasks:
> - (bind, [describe?,] execute) to reduce overhead of prepared
> statement execution (both in messages, as well as branches)
> - (anonymous parse, bind, describe, execute) to make it cheaper to
> send statements with out-of-line parameters
> - get rid of length limits of individual fields, probably w/ some variable
> length encoding (simple 7 bit?)
>

In preparation for the eventually-inevitable 64-bit field sizes, yes.

This should be on the protocol todo wiki.

> - allow *streaming* of large datums

Yes, very much +1 there. That's already on the wiki. Yeah:

* Permit lazy fetches of large values, at least out-of-line TOASTED values
http://www.postgresql.org/message-id/53FF0EF8.100@2ndquadrant.com

- type-level decisions about binary type transport, right now it's a lot
> of effort (including potentially additional roundtrips), to get the
> few important types to be transported in binary fashion. E.g. floating
> points are really expensive to stringify, bytea as text gets a lot
> bigger etc, but a lot of other types don't benefit a lot
>

Yeah, as distinct from now, where the client has specify param-by-param,
and where libpq doesn't support mixing text and binary formats in result
sets at all.

Again, needs wiki. I'll add.

- annotate COMMIT, PREPARE TRANSACTION, COMMIT PREPARED with LSN of
> associated WAL record
>

Already on the wiki, as is the related job of sending the xid of a txn to
the client when one is assigned.

> - have a less insane cancellation handling
>

+100

- nested table support
>
>
Can you elaborate on that one?

A few other points that come to mind for me are:

* labeled result sets (useful for stored procs, etc, as came up recently
with trying to figure out how to let stored procs have OUT params and
multiple result sets)

* room for other resultset formats later. Like Damir, I really want to add
protobuf or json serializations of result sets at some point, mainly so we
can return "entity graphs" in graph representation rather than left-join
projection.

* Robert Haas was talking about some issues relating to sync and the COPY
BOTH protocol a while ago, which we'd want to address.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services