Re: Introduce pg_receivewal gzip compression tests

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: gkokolatos(at)pm(dot)me
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Introduce pg_receivewal gzip compression tests
Date: 2021-07-13 08:14:05
Message-ID: YO1LTVgqpaY4jmN3@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Jul 13, 2021 at 04:37:53PM +0900, Michael Paquier wrote:
> Hmm. It looks like a waste in runtime once we mix LZ4 in that as that
> would mean 5 runs of pg_receivewal, but we really need only three of
> them with --endpos:
> - One with ZLIB compression.
> - One with LZ4 compression.
> - One without compression.
>
> Do you think that we could take advantage of what is now the only run
> of pg_receivewal --endpos for that? We could make the ZLIB checks run
> first, conditionally, and then let the last command with --endpos
> perform a full scan of the contents in $stream_dir with the .gz files
> already in place. The addition of LZ4 would be an extra conditional
> block similar to what's introduced in ZLIB, running before the last
> command without compression.

Poking at this problem, I partially take this statement back as this
requires an initial run of pg_receivewal --endpos to ensure the
creation of one .gz and one .gz.partial. So I guess that this should
be structured as:
1) Keep the existing pg_receivewal --endpos.
2) Add the ZLIB block, with one pg_receivewal --endpos.
3) Add at the end one extra pg_receivewal --endpos, outside of the ZLIB
block, which should check the creation of a .partial, non-compressed
segment. This should mention that we place this command at the end of
the test for the start streaming point computation.

LZ4 tests would be then between 2) and 3), or 1) and 2).
--
Michael

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message gkokolatos 2021-07-13 08:22:51 Re: Introduce pg_receivewal gzip compression tests
Previous Message Michael Paquier 2021-07-13 07:53:21 Re: Bogus HAVE_DECL_FOO entries in msvc/Solution.pm