Re: [REVIEW] Re: Compression of full-page-writes

On Tue, Aug 19, 2014 at 2:08 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2014-08-18 13:06:15 -0400, Robert Haas wrote:
>> On Mon, Aug 18, 2014 at 7:19 AM, Rahila Syed <rahilasyed90(at)gmail(dot)com> wrote:
>> >>According to the measurement result, the amount of WAL generated in
>> >>"Multiple Blocks in one run" than that in "Single Block in one run".
>> >>So ISTM that compression of multiple blocks at one run can improve
>> >>the compression ratio. Am I missing something?
>> >
>> > Sorry for using unclear terminology. WAL generated here means WAL that gets
>> > generated in each run without compression.
>> > So, the value WAL generated in the above measurement is uncompressed WAL
>> > generated to be specific.
>> > uncompressed WAL = compressed WAL + Bytes saved.
>> >
>> > Here, the measurements are done for a constant amount of time rather than
>> > fixed number of transactions. Hence amount of WAL generated does not
>> > correspond to compression ratios of each algo. Hence have calculated bytes
>> > saved in order to get accurate idea of the amount of compression in each
>> > scenario and for various algorithms.
>> >
>> > Compression ratio i.e Uncompressed WAL/compressed WAL in each of the above
>> > scenarios are as follows:
>> >
>> > Compression algo Multiple Blocks in one run Single Block in one run
>> >
>> > LZ4 1.21 1.27
>> >
>> > Snappy 1.19 1.25
>> >
>> > pglz 1.14 1.16
>> >
>> > This shows compression ratios of both the scenarios Multiple blocks and
>> > single block are nearly same for this benchmark.
>>
>> I don't agree with that conclusion. The difference between 1.21 and
>> 1.27, or between 1.19 and 1.25, is quite significant. Even the
>> difference beyond 1.14 and 1.16 is not trivial. We should try to get
>> the larger benefit, if it is possible to do so without an unreasonable
>> effort.
>
> Agreed.
>
> One more question: Do I see it right that multiple blocks compressed
> together compress *worse* than compressing individual blocks? If so, I
> have a rather hard time believing that the patch is sane.

Or the way of benchmark might have some problems.

Rahila,
I'd like to measure the compression ratio in both multiple blocks and
single block cases.
Could you tell me where the patch for "single block in one run" is?

Regards,

--
Fujii Masao