Re: Different compression methods for FPI

On Sat, Mar 13, 2021 at 08:48:33PM +0500, Andrey Borodin wrote:
> > 13 марта 2021 г., в 06:28, Justin Pryzby <pryzby(at)telsasoft(dot)com> написал(а):
> > Updated patch with a minor fix to configure.ac to avoid warnings on OSX.
> > And 2ndary patches from another thread to allow passing recovery tests.
> > Renamed to WAL_COMPRESSION_*
> > Split LZ4 support to a separate patch and support zstd. These show that
> > changes needed for a new compression method have been minimized, although not
> > yet moved to a separate, abstracted compression/decompression function.
>
> Thanks! Awesome work!
>
> > These two patches are a prerequisite for this patch to progress:
> > * Run 011_crash_recovery.pl with wal_level=minimal
> > * Make sure published XIDs are persistent
> >
> > I don't know if anyone will consider this patch for v14 - if not, it should be
> > set to v15 and revisit in a month.
>
> I want to note, that fixes for 011_crash_recovery.pl are not strictly necessary for this patch set.
> The problem in tests arises if we turn on wal_compression, absolutely independently from wal compression method.
> We turn on wal_compression in this test only for CI purposes.

I rearranged the patches to reflect this.
Change to zlib and zstd to level=1.
Add support for negative "zstd fast" levels.
Use correct length accounting for "hole" in LZ4 and ZSTD.
Fixed Solution.pm for zstd on windows.
Switch to zstd by default (for CI).
Add docs.

It occurred to me that the generic "compression API" might be of more
significance if we supported compression of all WAL (not just FPI). I imagine
that might use streaming compression.

--
Justin