Re: libpq compression (part 3)

From: Jacob Burroughs <jburroughs(at)instructure(dot)com>
To: Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: libpq compression (part 3)
Date: 2023-12-31 07:32:19
Message-ID: CACzsqT5-7xfbz+Si35TBYHzerNX3XJVzAUH9AewQ+Pp13fYBoQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> One thing I'm wondering: should it be possible for the client to change the compression it wants mid-connection? I can think of some scenarios where that would be useful to connection poolers: if a pooler does plain forwarding of the compressed messages, then it would need to be able to disable/enable compression if it wants to multiplex client connections with different compression settings over the same server connection.

I have reworked this patch series to make it easier to extend to
restart compression mid-connection once something in the vein of the
discussion in "Add new protocol message to change GUCs for usage with
future protocol-only GUCs" [1] happens. In particular, I have changed
the `CompressedMessage` protocol message to signal the current
compression algorithm any time the client should restart its streaming
decompressor and otherwise implicitly use whatever compression
algorithm and decompressor was used for previous `CompressedMessage` ,
which future work can leverage to trigger such a restart on update of
the client-supported compression algorithms.

> Option 2:
> This one is even cleaner in the common case but a bit worse in the
> uncommon case: just use one parameter and have
> compression/decompression enabling be part of the compression detail
> (e.g. "libpq_compression='gzip:no_de
> compress;lz4:level=2,no_compress;zstd'"
> or something like that, in which case the "none,gzip" case would
> become "'libpq_compression=gzip:no_compress'"). See
> https://www.postgresql.org/docs/current/app-pgbasebackup.html ,
> specifically the `--compress` flag, for how specifying compression
> algorithms and details works.

I ended up reworking this to use a version of this option in place of
the `none` hackery, but naming the parameters `compress` and
`decompress, so to disable compression but allow decompression you
would specify `libpq_compression=gzip:compress=off`.

Also my windows SSL test failures seem to have resolved themselves
with either these changes or a rebase, so I think things are truly in
a reviewable state now.

Attachment Content-Type Size
v3-0001-Add-IO-stream-abstraction.patch application/octet-stream 79.6 KB
v3-0002-Add-protocol-layer-compression-to-libpq.patch application/octet-stream 132.0 KB
v3-0003-Add-network-traffic-stats.patch application/octet-stream 22.7 KB
v3-0004-Add-basic-test-for-compression-functionality.patch application/octet-stream 6.9 KB
v3-0005-DO-NOT-MERGE-enable-compression-for-CI.patch application/octet-stream 3.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Paul Jungwirth 2023-12-31 08:51:49 Re: SQL:2011 application time
Previous Message Michael Paquier 2023-12-31 06:28:46 Re: pg_stat_statements: more test coverage