Re: Compress prune/freeze records with Delta Frame of Reference algorithm

Hello Andres,

> I'm unconvinced that this is a serious problem - typically the overhead of WAL
> volume due to pruning / freezing is due to the full page images emitted, not
> the raw size of the records. Once an FPI is emitted, this doesn't matter.
>
> What gains have you measured in somewhat realistic workloads?

So far, we have had no tests in any real production environment.
Moreover, the load in the new test
(recovery/t/052_prune_dfor_compression.pl) is quite contrived. However,
it demonstrates a compression ratio of more than 5, and it is measured
for an overall size of all prune/freeze records with no filtering.

Further development is the implementation of compression of unsorted
sequences. This is going to allow PostgreSQL to compress also the
'frozen' and the 'redirected' offset sequences, which should result in a
greater compression ratio.

But I agree with you, Andres, we need practical results to estimate a
profit. I wish we would test it on some real load soon.

Also I hope, independently of its usage in prune/freeze records, the
DFoR itself might be used for compression sequences in other places of PG.

Best regards,
Evgeny Voropaev.