Re: Protocol Compression (fourth attempt)

Hi Anthonin,

Thanks! Attached is a revision based on your v1. I kept your msg_buffer
refactoring as a separate patch, and retained PQcommMethods and decoded
message tracing. The second patch brings over client-to-server COPY
compression and reusable buffers from my prototype. The trace
checks use the existing libpq testclient.

As discussed, this version negotiates Zstandard at startup and
compresses DataRow/CopyData. Backend frames end at every ReadyForQuery,
frontend frames at CopyDone/CopyFail. Other codecs and levels, session
control, public thresholds and pg_stat_* counters are left for later.

On 30 Sep 2026, Anthonin Bonnefoy wrote:
> Maybe a combination of both (limited input size
> with at least x full messages) could work?

I used a 32 KiB input limit without a message-count minimum. Large
messages can span wrappers. Existing flush points still apply, so a
small query result does not wait for the buffer to fill.

In a local TCP comparison of 100000 rows with 80-byte payloads, the
first row arrived in about 0.5 ms with this revision versus 13 ms with
your v1.

You are right that libpq cannot deliver an incomplete row. Partial
flushing bounds buffering and allows compression, transfer and
decompression to overlap. A single large row still has to be decoded
in full. I would keep the threshold internal while we compare more
workloads.

Does this look like a useful common base?

Thank you!

Best regards, Andrey Borodin.