| From: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
|---|---|
| To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
| Cc: | Tomas Vondra <tomas(at)vondra(dot)me>, Andres Freund <andres(at)anarazel(dot)de>, Michael Paquier <michael(at)paquier(dot)xyz>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Amul Sul <sulamul(at)gmail(dot)com>, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, Anthonin Bonnefoy <anthonin(dot)bonnefoy(at)datadoghq(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: pg_waldump: support decoding of WAL inside tarfile |
| Date: | 2026-04-02 22:41:33 |
| Message-ID: | CA+hUKG+A-=BKP69ze5oc4O5RSBZtKk7dGoqYLeOUQSo8xugTyg@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Fri, Apr 3, 2026 at 7:48 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> *This* tar defaults to:
> --format=posix -f- -b20 --quoting-style=escape --rmt-command=/usr/bin/rmt
> --rsh-command=/usr/bin/ssh
From some quick googling, OpenSUSE is the only distro I could find
that does this, but the fine manual says they plan to make pax the
default upstream so eventually it might be everywhere:
https://www.gnu.org/software/tar/manual/html_section/Formats.html
> Interestingly, pg_verifybackup's t/003_corruption.pl test also fails
> with the same issue, so apparently this platform is even more
> aggressive about sparse-ifying files than Thomas' FreeBSD box.
Looks like sparse files and BTRFS might be a red herring then, if it's
simply been told to put a pax header on every file?
I think my system might be defaulting to "restricted pax", meaning
that pax headers are added to individual files only if necessary,
path-too-long etc, magic sparse format triggered, etc.
https://man.freebsd.org/cgi/man.cgi?query=libarchive-formats
> I'm inclined to push the logic for selecting these tar options
> into some common subroutine in Test::Utils, rather than having
> two copies (and maybe more later).
How about using --format=ustar, instead of that sparse control stuff?
That solves the BSD tar + ZFS problem and the OpenSUSE GNU tar default
format problem in one fell swoop, and actually says what we want
explicitly.
Should we have a <note> near the --path/--wal-path documentation to
state explicitly that usta format is required, and that pax is not
[yet] supported, and that --format=ustar might be required on some
systems?
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2026-04-02 22:50:50 | Re: pg_waldump: support decoding of WAL inside tarfile |
| Previous Message | David Rowley | 2026-04-02 22:20:50 | Re: Small and unlikely overflow hazard in bms_next_member() |