Re: pg_waldump: support decoding of WAL inside tarfile

From: Amul Sul <sulamul(at)gmail(dot)com>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: pg_waldump: support decoding of WAL inside tarfile
Date: 2026-03-18 15:16:35
Message-ID: CAAJ_b94fiDcsN6k27hV9772kNjEbtZG5ESQvXL1KQoUcRcxrGA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Mar 18, 2026 at 5:15 PM Amul Sul <sulamul(at)gmail(dot)com> wrote:
>
> On Wed, Mar 11, 2026 at 10:38 PM Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
> > [...]
> > Looks pretty good. I have squashed them into three patches I think are committable. Also attached is a diff showing what's changed - mainly this:
> >
> > . --follow + tar archive rejected (pg_waldump.c) — new validation prevents a confusing pg_fatal when combining --follow with a tar archive
> > . error messages split (archive_waldump.c) — the single "could not read file" error is now two distinct messages: "WAL segment is too short" (truncated file) vs "unexpected end of archive" (archive EOF) - Fixes an issue raised in review
> > . hash table cleanup (archive_waldump.c) — free_archive_reader now iterates and frees all remaining hash entries and destroys the table
> >
>
> The final squashed version looks good to me, thank you. But, I would
> like to propose splitting the 0001 patch into two separate commits: a
> preparatory refactoring of the pg_waldump code and a standalone commit
> that moves the tar archive detection and compression logic to a common
> location, as the latter is an independent improvement to the existing
> codebase. Additionally, since the test file refactoring was only kept
> separate to facilitate the review and has already been reviewed, I
> suggest merging those changes into the main feature patch i.e. 0002.
> All other elements should remain in a single preparatory refactoring
> patch for pg_waldump.
>
> Attached is the version that includes the proposed split. No
> additional changes to 0002 and 0003 patches.
>

Added the two missing 'Reviewed-by' lines to the credit section of the
commit message and did a minor optimization in get_archive_wal_entry.

Regards,
Amul

Attachment Content-Type Size
v19-0001-Move-tar-detection-and-compression-logic-to-comm.patch application/octet-stream 7.1 KB
v19-0002-pg_waldump-Preparatory-refactoring-for-tar-archi.patch application/octet-stream 8.4 KB
v19-0003-pg_waldump-Add-support-for-reading-WAL-from-tar-.patch application/octet-stream 54.7 KB
v19-0004-pg_verifybackup-Enable-WAL-parsing-for-tar-forma.patch application/octet-stream 16.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Sami Imseih 2026-03-18 15:20:56 Re: dshash_find_or_insert vs. OOM
Previous Message Lucas DRAESCHER 2026-03-18 15:11:13 [Bug Report + Patch] File descriptor leak when io_method=io_uring