Middleware Messages for FE/BE

From: Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Middleware Messages for FE/BE
Date: 2021-08-19 08:33:05
Message-ID: CANbhV-Ee88rz_a2_oWoYM4WsGZO-34VcJSr-T2xCM3ECKLa=rA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

The current FE/BE protocol assumes that the client is talking to the
server directly. If middleware wants to do something like load
balancing, the only current option is to inspect the incoming commands
and take action from that. That approach performs poorly and has
proven difficult to maintain, limiting the functionality in Postgres
ecosystem middleware.

It would be useful to have a way to speak to middleware within a
session or prior to each command. There are ways to frig this and
obviously it can always be done out-of-core, but there is a clear
requirement for various client tools to agree a standard way for them
to send messages to middleware rather than the database server. If we
get PostgreSQL Project approval for this, then authors of client and
middleware tools will know how to interoperate and can begin adding
features to take advantage of this, allowing the Postgres ecosystem to
improve and extend its middleware.

Byte1('M')
Identifies the message as a middleware message. (Or perhaps use 'U'
for User Message?).

Int32
Length of message contents in bytes, including self.

Int64
Routing/class identifier, allowing middleware to quickly ignore whole
classes of message if not appropriate. We would run some kind of
allocation scheme to ensure unique meaning, probably via the Postgres
Wiki. The first 2 billion values would be reserved for allocation by
the PostgreSQL Project itself, values beyond that open for external
allocation.

Byten
The message itself, where n is the remaining bytes in the message.

The message is intended for middleware only. The server always ignores
these messages, with an optional debug facility that can be enabled to
allow printing NOTICEs to allow testing.

I will supply a patch to any agreed message format, together with a
new libpq call to utilise this.

In summary: the patch is easy, the point is we need agreement to allow
and encourage interoperation between clients and middleware.

Thoughts?

--
Simon Riggs http://www.EnterpriseDB.com/

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message houzj.fnst@fujitsu.com 2021-08-19 09:04:22 RE: Skipping logical replication transactions on subscriber side
Previous Message houzj.fnst@fujitsu.com 2021-08-19 08:16:11 RE: Parallel Inserts (WAS: [bug?] Missed parallel safety checks..)