Re: Introduce pg_receivewal gzip compression tests

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: gkokolatos(at)pm(dot)me
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andrew Dunstan <andrew(at)dunslane(dot)net>
Subject: Re: Introduce pg_receivewal gzip compression tests
Date: 2021-07-19 07:03:33
Message-ID: YPUjxaIWkHUzBdwg@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Jul 16, 2021 at 02:08:57PM +0900, Michael Paquier wrote:
> This behavior is rather debatable, and it would be more instinctive to
> me to just skip any business related to the pre-padding if compression
> is enabled, at the cost of one extra callback in WalWriteMethod to
> grab the compression level (dir_open_for_write() skips that for
> compression) to allow receivelog.c to handle that. But at the same
> time few users are going to care about that as pg_receivewal has most
> likely always the same set of options, so complicating this code is
> not really appealing either.

I have chewed on that over the weekend, and skipping the padding logic
if we are in compression mode in open_walfile() makes sense, so
attached is a patch that I'd like to backpatch.

Another advantage of this patch is the handling of ".gz" is reduced to
one code path instead of four. That makes a bit easier the
introduction of new compression methods.

A second thing that was really confusing is that the name of the WAL
segment generated in this code path completely ignored the type of
compression. This led to one confusing error message if failing to
open a segment for write where we'd mention a .partial file rather
than a .gz.partial file. The versions of zlib I used on Windows
looked buggy so I cannot conclude there, but I am sure that this
should allow bowerbird to handle the test correctly.
--
Michael

Attachment Content-Type Size
receivewal-open.patch text/x-diff 7.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro Horiguchi 2021-07-19 07:15:36 Re: Failure with 004_logrotate in prairiedog
Previous Message Yugo NAGATA 2021-07-19 07:00:39 Re: corruption of WAL page header is never reported