Support long tar paths and link targets using PAX

From: Japin Li <japinli(at)hotmail(dot)com>
To: Pg Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Support long tar paths and link targets using PAX
Date: 2026-11-07 12:40:36
Message-ID: SY7PR01MB10921430D7F1623CF0B45C985B6752@SY7PR01MB10921.ausprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Hi hackers,

Currently, pg_basebackup can fail when creating plain-format backups if
a file path or symbolic link target cannot be represented in a
traditional ustar header.

The ustar format has fixed-size fields for member names and symbolic link
targets. Although long member names can be split between the `name` and
`prefix` fields, some paths still cannot be represented. Symbolic link
targets, on the other hand, are limited to 100 bytes.

This can be problematic when backing up tablespaces located in
directories with long paths, resulting in errors such as:

ERROR: symbolic link target too long for tar format:
file name "pg_tblspc/16386", target ".../007_tablespace/data/tblspc_djiS"

This patch adds support for POSIX PAX extended headers to handle these cases.

The implementation has the following changes:

- Add `tarCreateHeaderExt()` and `tarCreateHeaderExtSize()` to generate
tar headers with optional PAX extended headers.
- Prefer the traditional ustar representation whenever possible,
including splitting member names between the `name` and `prefix`
fields, so existing archives remain unchanged when their paths fit.
- Use the PAX `path` and `linkpath` attributes when member names or
symbolic link targets cannot be represented in a ustar header.
- Teach the astreamer tar parser to consume local PAX extended headers
and apply their attributes to the following archive member.
- Reject global PAX extended headers and ignore unknown attributes that
do not affect the properties handled by the parser.
- Update the tar archiver to reconstruct headers when PAX attributes need
to be preserved during re-archiving.
- Update the file extractor and pg_verifybackup to handle longer member
paths.

TAP tests are added to cover:

- Backups containing long file paths.
- Tablespaces with long paths and symbolic link targets.
- Verification of PAX-format backups using pg_verifybackup.
- Extraction of generated archives using an external tar utility.
- Re-archiving when pg_basebackup writes tar output to stdout.

Comments and reviews are welcome.

--
Regards,
Japin Li
ChengDu WenWu Information Technology Co., Ltd.

Attachment Content-Type Size
v1-0001-Support-long-tar-paths-and-link-targets-using-PAX.patch text/x-patch 53.0 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Xuneng Zhou 2026-11-07 13:05:56 Re: BGWORKER_INTERRUPTIBLE workers are never restarted after DROP/ALTER/CREATE DATABASE
Previous Message Bertrand Drouvot 2026-11-07 09:50:29 Re: Draft back-branch release notes are up