Re: astreamer fixes

On 2026-03-28 Sa 1:48 PM, Tom Lane wrote:
> I wrote:
>> 1. I do not think it's possible to get Z_STREAM_END from inflate()
>> in astreamer_gzip_decompressor_content, because we don't tell it
>> that we've reached end-of-stream.
> Nah, I'm mistaken about that. I forgot that a zlib-compressed stream
> has its own internal end-of-stream marker, and that's what triggers
> the Z_STREAM_END report.
>
>> 2. What Claude noticed I think, but failed to correct accurately, is
>> that astreamer_gzip_decompressor_finalize needs to invoke inflate()
>> with Z_FINISH, and pump it until it returns Z_STREAM_END.
> I spent some time testing this and could not devise a scenario where
> there is leftover data still to be decompressed once
> astreamer_gzip_decompressor_content finishes. Closer reading of
> zlib.h says that inflate() won't report all the input data as having
> been consumed if it runs out of output buffer space, so in practice
> the logic in astreamer_gzip_decompressor_content is fine, except that
> we're ignoring some error report codes that we shouldn't. So I think
> we should take the proposed patch hunk
>
> - if (res == Z_STREAM_ERROR)
> - pg_fatal("could not decompress data: %s", zs->msg);
> + if (res != Z_OK && res != Z_STREAM_END && res != Z_BUF_ERROR)
> + pg_fatal("could not decompress data: %s",
> + zs->msg ? zs->msg : "unknown error");
>
> but I don't like the other bit
>
> + /* If we've hit the end of the compressed stream, stop. */
> + if (res == Z_STREAM_END)
> + break;
>
> Paying attention to the amount of data consumed seems just as
> good and takes less code.
>

So IIUC, you agree with the patch except for this last piece, on which I
agree with your reasoning.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com