Re: Extensibility of the PostgreSQL wire protocol

On 2/19/21 8:48 AM, Heikki Linnakangas wrote:
> I can see value in supporting different protocols. I don't like the
> approach discussed in this thread, however.
>
> For example, there has been discussion elsewhere about integrating
> connection pooling into the server itself. For that, you want to have a
> custom process that listens for incoming connections, and launches
> backends independently of the incoming connections. These hooks would
> not help with that.

The two are not mutually exclusive. You are right that the current
proposal would not help with that type of built in connection pool, but
it may be extended to that.

Give the function, that postmaster is calling to accept a connection
when a server_fd is ready, a return code that it can use to tell
postmaster "forget about it, don't fork or do anything else with it".
This function is normally calling StreamConnection() before the
postmaster then forks the backend. But it could instead hand over the
socket to the pool background worker (I presume Jonah is transferring
them from process to process via UDP packet). The pool worker is then
launching the actual backends which receive a requesting client via the
same socket transfer to perform one or more transactions, then hand the
socket back to the pool worker.

All of that would still require a protocol extension that has special
messages for "here is a client socket for you" and "you can have that
back".

> I would recommend this approach: write a separate program that sits
> between the client and PostgreSQL, speaking custom protocol to the
> client, and libpq to the backend. And then move that program into a
> background worker process.

That is a classic protocol converting proxy. It has been done in the
past with not really good results, both performance wise as with respect
to protocol completeness.

Regards, Jan

--
Jan Wieck
Principle Database Engineer
Amazon Web Services