Re: [Patch]The Case For WAL-Logging pg_upgrade

This thread solves the same problem as mine (pg_upgrade_replica [1]):
resyncing a standby after pg_upgrade. We solve it in two different ways.
Worth comparing.

On Sat, Aug 8, 2026 at 6:11 AM John Naylor <johncnaylorls(at)gmail(dot)com> wrote:
> The transfer mode setting seems strange and not well motivated to me.

pg_upgrade_replica has a similar choice: where a tablespace goes on the
new standby (--tablespace-mapping). But this is a command-line flag, not
a WAL record. The operator picks a value once, when the tool runs. It
never has to mean the same thing again later, because there is no WAL
record to replay.

This is also why pg_upgrade_replica stays in src/bin/ and never touches
transam/ or rmgrdesc/. It reuses three things that already exist,
instead of adding a new WAL format:

- pg_upgrade's own manifest of unchanged files. This is a private
format. Only pg_upgrade_replica reads it. It is excluded from base
backups, because it describes the old cluster, not the cluster it is
stored in.

- The forged manifest pg_upgrade_replica builds for pg_basebackup. This
is a real backup_manifest, read by the same code
(common/parse_manifest.c) that pg_basebackup and pg_combinebackup
already use.

- The block-level diff. This comes from pg_basebackup --incremental's
own WAL-summary code, already in core.

Outside of that excludeFiles line, pg_upgrade also writes its own small
manifest file into the new cluster's data directory. That file is
core's only new footprint; nothing else in the backend's WAL or redo
code changes. The src/bin/ tool itself is easy to delete if it turns
out to be the wrong answer.

> For rollback, can't the operator just pause the rollback target at the
> handoff checkpoint while still on the old binary and promote if
> necessary? Am I missing something?

In my design, yes: --old-replica is read-only for the whole run, so it
stays intact and bootable on the old binary even after the new standby
starts. That does not prove the transfer-mode GUC is unneeded, it just
describes how my tool happens to behave by default. --link trades that
property away on purpose, to save disk space, the same tradeoff
pg_upgrade's own --link already makes.

Bohyun's design also fixes something mine does not: the gap between an
upgrade and the next base backup. pg_upgrade_replica only runs after the
upgrade, against an already-running new primary. It says nothing about a
primary that crashes before anyone takes a new backup or resyncs a
standby. WAL-logging the upgrade fixes this for free, because recovery
becomes normal WAL replay across the upgrade boundary.

The handoff-then-pause step also answers one question for free: was the
standby caught up to the exact checkpoint before the upgrade? It answers
this through normal WAL replay, before the pause record is even
reached. My tool has to check this after the fact, by comparing
--old-replica's pg_control to the manifest. It has no other way to
check, because it works outside the replication stream.

Two questions for Bohyun, based on the patch as posted:

- WAL size. XLOG_UPGRADE_RELFILE_DATA and XLOG_UPGRADE_SLRU_DATA write a
full copy of every changed catalog and SLRU block. For a cluster with
many databases, or a large catalog, how big does this get? How does it
compare to a plain pg_upgrade --link run, which costs almost nothing
today? Has this been tested on something bigger than the TAP tests?

- The RELINK ENOENT case. In the XLOG_UPGRADE_RELINK redo, if stat()
cannot find a source file under pg_upgrade_standby_old_datadir, the
code just skips it (continue on ENOENT). Is this only reachable for
files that are correctly absent, like an unlogged relation's main fork
(the same case I have to handle)? Or could a truly missing file, from
a wrong path or a damaged old datadir, also hit this same branch, and
leave the standby silently missing a relation file instead of failing
loudly?

I see these as two different, valid answers to the same problem, not as
something to merge. I wanted the comparison on record.

[1]
https://www.google.com/url?q=https://www.postgresql.org/message-id/flat/CA%252BnrD2fqdeEJkGJrDt%252B-a7Uqr4OucXZHvSVxLCb9J0EkN%252BhLhw%2540mail.gmail.com&source=gmail&ust=1786631006795000&sa=E

Marco Nenciarini
EnterpriseDB