Re: pg_waldump: support decoding of WAL inside tarfile

From: Amul Sul <sulamul(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
Date: 2025-08-26 11:52:20
Message-ID: CAAJ_b97JA8ehy_UDddrnGwDt9HG5NmJq8ATtmeMqo7YD-=tLyQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Aug 25, 2025 at 5:58 PM Amul Sul <sulamul(at)gmail(dot)com> wrote:
>
> On Thu, Aug 7, 2025 at 7:47 PM Amul Sul <sulamul(at)gmail(dot)com> wrote:
> > [....]
> > -----------------------------------
> > Known Issues & Status:
> > -----------------------------------
> > - Timeline Switching: The current implementation in patch 006 does not
> > correctly handle timeline switching. This is a known issue, especially
> > when a timeline change occurs on a WAL file that has been written to a
> > temporary location.
> >
>
> This is still pending and will be addressed in the next version.
> Therefore, patch 0006 remains marked as WIP.
>

After testing pg_waldump, I have realised that my previous
understanding of its timeline handling was incorrect. I had mistakenly
assumed by reading xlogreader code that it would use the same
timeline-switching logic found in xlogreader, without first verifying
this behavior. In testing, I found that pg_waldump does not follow
timeline switches. Instead, it expects all WAL files to be from a
single timeline, which is either specified by the user or determined
from the starting segment or default 1.

This is a positive finding, as it means we don't need to make
significant changes to align pg_waldump's current behavior. The
attached patches are now complete and no longer works in progress --
read for review. Additionally, I've dropped patch v2-0004 because it is
no longer necessary. The primary patches that implement the proposed
feature are now 0004 and 0005 in the attached set.

Regards,
Amul

Attachment Content-Type Size
v3-0001-Refactor-pg_waldump-Move-some-declarations-to-new.patch application/x-patch 2.2 KB
v3-0002-Refactor-pg_waldump-Separate-logic-used-to-calcul.patch application/x-patch 2.3 KB
v3-0003-Refactor-pg_waldump-Restructure-TAP-tests.patch application/x-patch 5.5 KB
v3-0004-pg_waldump-Add-support-for-archived-WAL-decoding.patch application/x-patch 34.5 KB
v3-0005-pg_waldump-Remove-the-restriction-on-the-order-of.patch application/x-patch 17.5 KB
v3-0006-pg_verifybackup-Delay-default-WAL-directory-prepa.patch application/x-patch 1.7 KB
v3-0007-pg_verifybackup-Rename-the-wal-directory-switch-t.patch application/x-patch 15.6 KB
v3-0008-pg_verifybackup-enabled-WAL-parsing-for-tar-forma.patch application/x-patch 9.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message xx Z 2025-08-26 11:54:10 How to configure client-side TLS ciphers for streaming replication?
Previous Message Fujii Masao 2025-08-26 11:43:45 Re: pg_restore --no-policies should not restore policies' comment