Re: Compression of full-page-writes

On Wed, Oct 23, 2013 at 7:05 AM, KONDO Mitsumasa
<kondo(dot)mitsumasa(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> (2013/10/22 12:52), Fujii Masao wrote:
>>
>> On Tue, Oct 22, 2013 at 12:47 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
>> wrote:
>>>
>>> On Mon, Oct 21, 2013 at 4:40 PM, KONDO Mitsumasa
>>> <kondo(dot)mitsumasa(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>>>>
>>>> (2013/10/19 14:58), Amit Kapila wrote:
>>>>>
>>>>> On Tue, Oct 15, 2013 at 11:41 AM, KONDO Mitsumasa
>>>>> <kondo(dot)mitsumasa(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>>>> In
>>>> actual, many of NoSQLs use snappy for purpose of higher performance.
>>>
>>>
>>> Okay, you can also check the results with snappy algorithm, but don't
>>> just rely completely on snappy for this patch, you might want to think
>>> of another alternative for this patch.
>>
>>
>> So, our consensus is to introduce the hooks for FPW compression so that
>> users can freely select their own best compression algorithm?
>
> Yes, it will be also good for future improvement. But I think WAL
> compression for disaster recovery system should be need in walsender and
> walreceiver proccess, and it is propety architecture for DR system. Higher
> compression ratio with high CPU usage algorithm in FPW might affect bad for
> perfomance in master server.

This is true, thats why there is a discussion for pluggable API for
compression of WAL, we should try to choose best algorithm from the
available choices. Even after that I am not sure it works same for all
kind of loads, so user will have option to completely disable it as
well.

> If we can set compression algorithm in
> walsender and walreciever, performance is same as before or better, and WAL
> send performance will be better.

Do you mean to say that walsender should compress the data before
sending and then walreceiver will decompress it, if yes then won't it
add extra overhead on standby, or do you think as walreceiver has to
read less data from socket, so it will compensate for it. I think may
be we should consider this if the test results are good, but lets not
try to do this until the current patch proves that such mechanism is
good for WAL compression.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com