Re: pg_waldump: support decoding of WAL inside tarfile

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Amul Sul <sulamul(at)gmail(dot)com>, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
Date: 2026-03-21 12:21:49
Message-ID: 1663e08f-2711-42ed-a3c7-08632c852911@dunslane.net
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


On 2026-03-21 Sa 2:34 AM, Tom Lane wrote:
> Michael Paquier<michael(at)paquier(dot)xyz> writes:
>> On Fri, Mar 20, 2026 at 11:49:02PM -0400, Tom Lane wrote:
>>> Buildfarm members batta and hachi don't like this very much.
>> I did not look at what's happening on the host, but it seems like a
>> safe bet to assume that we are not seeing many failures in the
>> buildfarm because we don't have many animals that have the idea to add
>> --with-zstd to their build configuration, like these two ones.
> That may be part of the story, but only part. I spent a good deal of
> time trying to reproduce batta & hachi's configurations locally, on
> several different platforms, but still couldn't duplicate what they
> are showing.
>
>

Yeah, I haven't been able to reproduce it either. But while
investigating I found a couple of issues. We neglected to add one of the
tests to meson.build, and we neglected to close some files, causing
errors on windows.

I also noticed a possible bug in astreamer, where the decompressor
finalize functions send bbs_buffer.maxlen bytes to the next streamer
when flushing remaining data at end-of-stream. This seems wrong because
the buffer may only be partially filled with valid decompressed data.
Possible patch for that attached. (But I don't think it's related to
these failures).

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

Attachment Content-Type Size
001-Fix-pg_waldump-archive-reader-file-handle-leak-and-r.patch text/x-patch 895 bytes
002-Fix-astreamer-decompressor-finalize-to-send-correct-.patch text/x-patch 2.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ed Behn 2026-03-21 12:22:38 Persistent data across SETOF calls
Previous Message Etsuro Fujita 2026-03-21 11:56:41 Re: Use-after-free issue in postgres_fdw