Re: libpq compression

Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> writes:

> On 22.06.2018 18:59, Robbie Harwood wrote:
>> Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> writes:
>>> On 21.06.2018 20:14, Robbie Harwood wrote:
>>>> Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> writes:
>>>>> On 21.06.2018 17:56, Robbie Harwood wrote:
>>>>>> Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> writes:
>>>>>>> On 20.06.2018 23:34, Robbie Harwood wrote:
>>>>>>>> Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> writes:
>>>>>>>>
>>>>>>>>> My idea was the following: client want to use compression. But
>>>>>>>>> server may reject this attempt (for any reasons: it doesn't
>>>>>>>>> support it, has no proper compression library, do not want to
>>>>>>>>> spend CPU for decompression,...) Right now compression
>>>>>>>>> algorithm is hardcoded. But in future client and server may
>>>>>>>>> negotiate to choose proper compression protocol. This is why
>>>>>>>>> I prefer to perform negotiation between client and server to
>>>>>>>>> enable compression.
>>>>>>>>
>>>>>>>> Well, for negotiation you could put the name of the algorithm
>>>>>>>> you want in the startup. It doesn't have to be a boolean for
>>>>>>>> compression, and then you don't need an additional round-trip.
>>>>>>>
>>>>>>> Sorry, I can only repeat arguments I already mentioned:
>>>>>>>
>>>>>>> - in future it may be possible to specify compression algorithm
>>>>>>>
>>>>>>> - even with boolean compression option server may have some
>>>>>>> reasons to reject client's request to use compression
>>>>>>>
>>>>>>> Extra flexibility is always good thing if it doesn't cost too
>>>>>>> much. And extra round of negotiation in case of enabling
>>>>>>> compression seems to me not to be a high price for it.
>>>>>>
>>>>>> You already have this flexibility even without negotiation. I
>>>>>> don't want you to lose your flexibility. Protocol looks like
>>>>>> this:
>>>>>>
>>>>>> - Client sends connection option "compression" with list of
>>>>>> algorithms it wants to use (comma-separated, or something).
>>>>>>
>>>>>> - First packet that the server can compress one of those algorithms
>>>>>> (or none, if it doesn't want to turn on compression).
>>>>>>
>>>>>> No additional round-trips needed.
>>>>>
>>>>> This is exactly how it works now... Client includes compression
>>>>> option in connection string and server replies with special
>>>>> message ('Z') if it accepts request to compress traffic between
>>>>> this client and server.
>>>>
>>>> No, it's not. You don't need this message. If the server receives
>>>> a compression request, it should just turn compression on (or not),
>>>> and then have the client figure out whether it got compression
>>>> back.
>>>
>>> How it will managed to do it. It receives some reply and first of
>>> all it should know whether it has to be decompressed or not.
>>
>> You can tell whether a message is compressed by looking at it. The
>> way the protocol works, every message has a type associated with it:
>> a single byte, like 'R', that says what kind of message it is.
>
> Compressed message can contain any sequence of bytes, including 'R':)

Then tag your messages with a type byte. Or do it the other way around
- look for the zstd framing within a message.

Please, try to work with me on this instead of fighting every design
change.

Thanks,
--Robbie