Re: Proposal: http2 wire format

On 26 March 2018 at 17:01, Damir Simunic <damir(dot)simunic(at)wa-research(dot)ch>
wrote:

>
> > - Doesn't break new clients connecting to old servers
> >
>
> Old server sends “Invalid startup packet” and closes the connection;
> client’s TLS layer reports an error. Does that count as not breaking new
> clients?
>
>
libpq would have to do something like it does now for ssl connections,
falling back to non-ssl, and offering a connection option to make it try
the v3 protocol immediately without bothering with v4.

> > - No extra round trips for new client -> old server . I don't personally
> care about old client -> new server so much, but should be able to offer a
> pg_hba.conf option to ensure v3 proto only or otherwise prevent extra round
> trips in this case too.
>
> Can we talk about this more, please?
>

As above. A newer libpq should not perform worse on an existing server than
an older libpq.

>
> Check.
>
> Extensibility is the essence of h2, we’re getting this for free.
>
>
Please elaborate somewhat for people not already strongly familiar with
HTTP2.

BTW, please stop saying "h2" when you mean HTTP2. It's really confusing,
because I keep thinking you are talking about H2, the database engine (
http://www.h2database.com/), which has PostgreSQL protocol and syntax
compatibility as well as its own wire protocol.

> > - Has a wireshark dissector
>
> Check.
>

... including understanding of the PostgreSQL bits that are payload within
the protocol.

Look at what the current dissector does - capture some packets.

>
> >
> > - Is practical to implement in connection pooler proxies like pgbouncer,
> pgpool
>
> Something I’m planning to look into and address.
>
> New connection poolers might become feasible, too: nginx, nghttpx, etc.
> (for non-web related scenarios as well). Opting into h2 lets us benefit
> from a much larger amount of time and resources being spent on improving
> things that matter. Reverse proxies face the same architectural challenges
> as pg-only connection poolers do.
>
>
... which is nice, but doesn't change the fact that a protocol revision
that completely and unfixably breaks existing tools much of the community
relies on won't go far.

> > - Any libraries used are widespread enough that they're present in at
> least RHEL7 and Debian Stable. We *can't* just bundle extras in our
> sources, and packagers are unlikely to be at all happy packaging an extra
> lib or backport for us. They'll probably just disable the new protocol.
>
> Check.
>
> Let me see if I can make a table showing parallel availability of Postgres
> and libnghttp versions on mainstream platforms. If there are any gaps, I’m
> sure it is possible to lobby for inclusion of libnghttp where it matters. I
> see Debian has it for wheezy, jessie, and sid, while pg10 is on sid and
> buster.
>
>
Good plan. But be clear that this is super experimental.

> >
> > - No regressions for support of SASL / SCRAM, GSSAPI, TLS with X.509
> client certs, various other auth methods.
> >
>
> Check.
>
> Adding new auth method keyword (“h2”) in pg_hba will give us a clean code
> path to work with.
>

I think you missed the point there entirely.

HTTP2 isn't an authentication method. It's a wire protocol. It will be
necessary to support authentication methods including, but not limited to,
GSSAPI, SSPI (windows), SCRAM, etc *on any new protocol*.

If you propose a new protocol, to replace the v3 protocol, and it doesn't
support SSPI or SCRAM I rate your chances as about zero of getting serious
interest. You'll be back in extension-for-webdevs town.

>
> > Now, a protocol that cannot satisfy these is IMO not a complete
> non-starter. It just has to be treated as an optional feature to help out
> webapps, with quite different design criteria as a result, and cannot be
> allowed to be as intrusive. Where changes to core protocol logic paths are
> required it'd have to add plugin mechanisms/hooks instead of adding its own
> new logic directly.
>
> While web-related scenarios are the first thing that comes to ming when
> talking about h2, (and that should not be disregarded), this proposal looks
> at the bigger picture of future-proofing the protocol.
> Headers/data/trailers split, and feature/ content negotiation are far
> bigger benefits then being web friendly.
>

You mentioned something about bundling queries in the startup packet.
That's cool if your queries don't need to adapt to server version etc,
which will often be the case. But doesn't that imply rather high backend
startup/shutdown costs?

There's a reason everyone with high rates of small simple queries uses
poolers right now.

Such a protocol would help poolers a lot, but not gain a great deal for the
core server without some kind of backend pooling, which is a huge separate
topic.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services