Re: Teach pg_receivewal to use lz4 compression

On Thu, Nov 04, 2021 at 04:31:48PM +0900, Michael Paquier wrote:
> I would have expected read_size to be (readend - readp) to match with
> the remaining data in the read buffer that we still need to read.
> Shouldn't out_size also be LZ4_CHUNK_SZ to match with the size of the
> output buffer where all the contents are read? By setting it to 1, I
> think that this is doing more loops into LZ4F_decompress() than really
> necessary. It would not hurt either to memset(0) those buffers before
> they are used, IMO. I am not completely sure either, but should we
> use the number of bytes returned by LZ4F_decompress() as a hint for
> the follow-up loops?
>
> +#ifdef HAVE_LIBLZ4
> + while (readp < readend)
> + {
> + size_t read_size = 1;
> + size_t out_size = 1;
> +
> + status = LZ4F_decompress(ctx, outbuf, &out_size,
> + readbuf, &read_size, NULL);

And... It happens that the error from v9 is here, where we need to
read the amount of remaining data from "readp", and not "readbuf" :)

It is already late here, I'll continue on this stuff tomorrow, but
this looks rather committable overall.
--
Michael