Re: Different compression methods for FPI

On Sun, Mar 14, 2021 at 07:31:35PM -0500, Justin Pryzby wrote:
> On Sat, Mar 13, 2021 at 08:48:33PM +0500, Andrey Borodin wrote:
> > > 13 марта 2021 г., в 06:28, Justin Pryzby <pryzby(at)telsasoft(dot)com> написал(а):
> > > Updated patch with a minor fix to configure.ac to avoid warnings on OSX.
> > > And 2ndary patches from another thread to allow passing recovery tests.
> > > Renamed to WAL_COMPRESSION_*
> > > Split LZ4 support to a separate patch and support zstd. These show that
> > > changes needed for a new compression method have been minimized, although not
> > > yet moved to a separate, abstracted compression/decompression function.
> >
> > Thanks! Awesome work!
> >
> > > These two patches are a prerequisite for this patch to progress:
> > > * Run 011_crash_recovery.pl with wal_level=minimal
> > > * Make sure published XIDs are persistent
> > >
> > > I don't know if anyone will consider this patch for v14 - if not, it should be
> > > set to v15 and revisit in a month.
> >
> > I want to note, that fixes for 011_crash_recovery.pl are not strictly necessary for this patch set.
> > The problem in tests arises if we turn on wal_compression, absolutely independently from wal compression method.
> > We turn on wal_compression in this test only for CI purposes.
>
> I rearranged the patches to reflect this.
> Change to zlib and zstd to level=1.
> Add support for negative "zstd fast" levels.
> Use correct length accounting for "hole" in LZ4 and ZSTD.
> Fixed Solution.pm for zstd on windows.
> Switch to zstd by default (for CI).
> Add docs.

Changes:
- Allocate buffer sufficient to accommodate any supported compression method;
- Use existing info flags argument rather than adding another byte for storing
the compression method; this seems to be what was anticipated by commit
57aa5b2bb and what Michael objected to.

I think the existing structures are ugly, so maybe this suggests using a GUC
assign hook to support arbitrary compression level, and maybe other options.

--
Justin