Re: Zstandard support for toast compression

On Tue, May 17, 2022 at 02:54:28PM -0400, Robert Haas wrote:
> I don't particularly have anything against adding Zstandard
> compression here, but I wonder whether there's any rush. If we decide
> not to add this now, we can always change our minds and add it later,
> but if we decide to add it now, there's no backing it out. I'd
> probably be inclined to wait and see if our public demands it of us.

+1

One consideration is that zstd with negative compression levels is comparable
to LZ4, and with positive levels gets better compression. It can serve both
purposes (oltp vs DW or storage-limited vs cpu-limited).

If zstd is supported, then for sure at least its compression level should be
configurable. default_toast_compression should support it.
https://commitfest.postgresql.org/35/3102/

Also, zstd is a few years newer than lz4. Which I hope means that the API is a
bit better/further advanced - but (as we've seen) may still be evolving.

Zstd allows some of its options to be set by environment variable - in
particular, the number of threads. We should consider explicitly setting
that to zero in the toast context unless we're convinced it's no issue for
every backend (not just basebackup).

--
Justin