Re: [REVIEW] Re: Compression of full-page-writes

On Thu, Sep 11, 2014 at 09:37:07AM -0300, Arthur Silva wrote:
> I agree that there's no reason to fix an algorithm to it, unless maybe it's
> pglz. There's some initial talk about implementing pluggable compression
> algorithms for TOAST and I guess the same must be taken into consideration
> for the WAL.
>
> --
> Arthur Silva
>
>
> On Thu, Sep 11, 2014 at 2:46 AM, Rahila Syed <rahilasyed(dot)90(at)gmail(dot)com>
> wrote:
>
> > >I will repeat the above tests with high load on CPU and using the
> > benchmark
> > given by Fujii-san and post the results.
> >
> > Average % of CPU usage at user level for each of the compression algorithm
> > are as follows.
> >
> > Compression Multiple Single
> >
> > Off 81.1338 81.1267
> > LZ4 81.0998 81.1695
> > Snappy: 80.9741 80.9703
> > Pglz : 81.2353 81.2753
> >
> > <
> > http://postgresql.1045698.n5.nabble.com/file/n5818552/CPU_utilization_user_single.png
> > >
> > <
> > http://postgresql.1045698.n5.nabble.com/file/n5818552/CPU_utilization_user.png
> > >
> >
> > The numbers show CPU utilization of Snappy is the least. The CPU
> > utilization
> > in increasing order is
> > pglz > No compression > LZ4 > Snappy
> >
> > The variance of average CPU utilization numbers is very low. However ,
> > snappy seems to be best when it comes to lesser utilization of CPU.
> >
> > As per the measurement results posted till date
> >
> > LZ4 outperforms snappy and pglz in terms of compression ratio and
> > performance. However , CPU utilization numbers show snappy utilizes least
> > amount of CPU . Difference is not much though.
> >
> > As there has been no consensus yet about which compression algorithm to
> > adopt, is it better to make this decision independent of the FPW
> > compression
> > patch as suggested earlier in this thread?. FPW compression can be done
> > using built in compression pglz as it shows considerable performance over
> > uncompressed WAL and good compression ratio
> > Also, the patch to compress multiple blocks at once gives better
> > compression
> > as compared to single block. ISTM that performance overhead introduced by
> > multiple blocks compression is slightly higher than single block
> > compression
> > which can be tested again after modifying the patch to use pglz . Hence,
> > this patch can be built using multiple blocks compression.
> >
> > Thoughts?
> >

Hi,

The big (huge) win for lz4 (not the HC variant) is the enormous compression
and decompression speed. It compresses quite a bit faster (33%) than snappy
and decompresses twice as fast as snappy.

Regards,
Ken