Re: libpq compression

On Fri, Jun 22, 2012 at 12:38 PM, Euler Taveira <euler(at)timbira(dot)com> wrote:
> On 20-06-2012 17:40, Marko Kreen wrote:
>> On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug <fgp(at)phlo(dot)org> wrote:
>>> I'm starting to think that relying on SSL/TLS for compression of
>>> unencrypted connections might not be such a good idea after all. We'd
>>> be using the protocol in a way it quite clearly never was intended to
>>> be used...
>>
>> Maybe, but what is the argument that we should avoid
>> on encryption+compression at the same time?
>>
>> AES is quite lightweight compared to compression, so should
>> be no problem in situations where you care about compression.
>>
> If we could solve compression problem without AES that will turn things
> easier. Compression-only via encryption is a weird manner to solve the problem
> in the user's POV.
>
>> RSA is noticeable, but only for short connections.
>> Thus easily solvable with connection pooling.
>>
> RSA overhead is not the main problem. SSL/TLS setup is.
>
>> And for really special compression needs you can always
>> create a UDF that does custom compression for you.
>>
> You have to own the code to modify it; it is not always an option.
>
>> So what exactly is the situation we need to solve
>> with postgres-specific protocol compression?
>>
> Compression only support. Why do I need to set up SSL/TLS just for compression?
>
> IMHO SSL/TLS use is no different from relying in another library to handle
> compression for the protocol and more it is compression-specific. That way, we
> could implement another algorithms in such library without needing to modify
> libpq code. Using SSL/TLS you are bounded by what SSL/TLS software products
> decide to use as compression algorithms. I'll be happy to maintain the code
> iif it is postgres-specific or even as close as possible to core.

I guess my feeling on this is that, so far as I can see, supporting
compression via OpenSSL involves work and trade-offs, and supporting
it without depending on OpenSSL also involves work, and trade-offs.
So it's not real evident to me that we should prefer one to the other
on general principle. It seems to me that a lot might come down to
performance. If someone can demonstrate that using an external
library involves gets significantly better compression, chews up
significantly less CPU time, and/or is significantly less code than
supporting this via OpenSSL, then maybe we ought to consider it.
OpenSSL is kind of an ugly piece of code, and all things being equal
depending on it more heavily would not be my first choice.

On the other hand, this does not seem to me to be a situation where we
should accept a patch to use an external library just because someone
takes the time to write one, because there is also a non-trivial cost
for the entire project to depending on more things; or if the
compression code gets put into the backend, then there's a cost to us
to maintain that code inside our source base. So I think we really
need someone to try this both ways and compare. Right now it seems
like we're mostly speculating on how well either approach would work,
which is not as good as having some experimental results. If, for
example, someone can demonstrate that an awesomebsdlz compresses 10x
as fast as OpenSSL... that'd be pretty compelling.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company