Re: libpq compression

On 05.06.2018 09:04, Thomas Munro wrote:
> On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik
> <k(dot)knizhnik(at)postgrespro(dot)ru> wrote:
>> Concerning specification of compression level: I have made many experiments
>> with different data sets and both zlib/zstd and in both cases using
>> compression level higher than default doesn't cause some noticeable increase
>> of compression ratio, but quite significantly reduce speed. Moreover, for
>> "pgbench -i" zstd provides better compression ratio (63 times!) with
>> compression level 1 than with with largest recommended compression level 22!
>> This is why I decided not to allow user to choose compression level.
> Speaking of configuration, are you planning to support multiple
> compression libraries at the same time? It looks like the current
> patch implicitly requires client and server to use the same configure
> option, without any attempt to detect or negotiate. Do I guess
> correctly that a library mismatch would produce an incomprehensible
> corrupt stream message?
>
Frankly speaking I am not sure that support of multiple compression
libraries at the same time is actually needed.
If we build Postgres from sources, then both frontend and backend
libraries will use the same compression library.
zlib is available almost everywhere and Postgres in any case is using it.
zstd is faster and provides better compression ratio. So in principle it
may be useful to try first to use zstd and if it is not available then
use zlib.
It will require dynamic loading of this libraries.

libpq stream compression is not the only place where compression is used
in Postgres. So I think that the problem of  choosing compression algorithm
and supporting custom compression methods should be fixed at some upper
level. There is patch for custom compression method at commit fest.
May be it should be combined with this one.

Right now if client and server libpq libraries were built with different
compression libraries, then decompress error will be reported.
Supporting multiple compression methods will require more sophisticated
handshake protocol so that client and server can choose compression
method which is supported by both of them.
But certainly it can be done.

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company