Re: [Patch]The Case For WAL-Logging pg_upgrade

> I think my phrasing was ambiguous: When I said "the system must know
what file paths to stick into the WAL stream", I didn't mean creating
path strings, I meant the enumeration done by capture_all_relfiles()
etc. Normally, a WAL record enters the stream because a logged write
happens. In the patch, the input to WAL comes from a directory walk of
already-written files. If this walk misses anything, then the standby
diverges from the primary.

I believe this can be prevented by failing loudly on any mismatch rather
than silently skipping unclassifiable filenames, which I am planning to
include in the next patch version.

> Also, the result of RELINK depends on a
GUC, so IIUC two standbys can receive the same WAL and end up with
different results. That's not good.

The GUCs are introduced because we cannot assume the standby's storage
layout exactly matches the primary's, neither where the retained
pre-upgrade directory sits, nor how its files are physically placed.
It also depends on how the cluster intends to use the standby. For
instance, if the operator wants to keep the standby as a rollback target,
it may be worthwhile to use a different transfer mode via the newly
introduced pg_upgrade_standby_transfer_mode GUC, which I believe is useful.
Nevertheless, the reconstructed cluster will be logically identical, even
if the physical representation diverges.

> I need to make sure my orchestration story is on solid ground before
sharing anything publicly.

I look forward to hearing more. Understanding how others would orchestrate
pg_upgrade would help me shape the interface toward something more
general-purpose.

Regards,
Bohyun

On Wed, Aug 5, 2026 at 7:15 AM John Naylor <johncnaylorls(at)gmail(dot)com> wrote:

> On Mon, Aug 3, 2026 at 4:01 PM Bohyun Lee <bohyun(dot)lee(at)databricks(dot)com>
> wrote:
>
> > The RELINK WAL record also doesn't contain paths: entries carry
> (tablespace_oid, database_oid, relfilenumber, forknum, segno), and redo
> derives the path in the same form as xl_dbase_create_file_copy_rec. Only
> DIRTREE and RAWFILE carry paths.
>
> I think my phrasing was ambiguous: When I said "the system must know
> what file paths to stick into the WAL stream", I didn't mean creating
> path strings, I meant the enumeration done by capture_all_relfiles()
> etc. Normally, a WAL record enters the stream because a logged write
> happens. In the patch, the input to WAL comes from a directory walk of
> already-written files. If this walk misses anything, then the standby
> diverges from the primary. Also, the result of RELINK depends on a
> GUC, so IIUC two standbys can receive the same WAL and end up with
> different results. That's not good.
>
> > I take the layering concern to be the more substantive one, and I don't
> think "the primitives are upstream's" answers it.
>
> On the odd chance you are using an LLM to write emails, I suggest
> refraining. While on the subject of list etiquette, please see the
> following on top- versus bottom-posting:
>
> https://en.wikipedia.org/wiki/Posting_style#Bottom-posting
>
> > Could you say more about the other options you have in mind and share
> your prototype if it's shareable? I'd be glad to work in that direction if
> that would provide a better foundation.
>
> I need to make sure my orchestration story is on solid ground before
> sharing anything publicly. One thing that hadn't crossed my mind yet
> is the possibility of getting PITR to work through an upgrade
> boundary. That's what I found most compelling about the presentation
> of your patch, and I'll have to keep that in mind as I proceed.
>
> --
> John Naylor
> Amazon Web Services
>