Re: [PATCH] pg_upgrade: add --initdb option to create the new cluster automatically

On 14/10/2026 11:32, Bohyun Lee wrote:
>> Why not write the logs under the new data directory, just like without
>> --initdb?
>
> Agreed, the normal logs should end up in the new datadir, as they do
> without --initdb.
>
> The patch captures early server and initdb output under
>
> ./pg_upgrade_output.d/<timestamp>/ in the current working directory,
>
> because initdb requires an empty target directory.
>
> But once initdb succeeds, it removes the temporary logs unless --retain
> is specified.
>
> Subsequent output is written under
>
> <new datadir>/pg_upgrade_output.d/<timestamp>/ using the existing
> cleanup and --retain handling.
>
> If initdb fails, pg_upgrade reports initialization failure and points to
> the retained log.

That sounds weird. To find the logs, you have to check both places; they
might be under ./pg_upgrade_output.d/ or <new
datadir>/pg_upgrade_output.d/ depending on the stage that it failed at.
Also, if initdb succeeds but the operation fails later, there's no trace
of the successful initdb in <new datadir>/pg_upgrade_output.d/, those
logs are lost even though the logs for other stages are there.

I see why you did it that way; there's a chicken-and-egg problem. In
order to log the initdb invocation, the log dir must be created before
initdb is launched, but initdb errors out if the target directory is not
empty.

We could modify initdb so that it ignores "pg_upgrade_output.d" and
considers the target dir as empty as long as nothing else exists.
Another idea is to run initdb in a subdir first, and rename it into the
right place later (see below).

>> But then you need write access to the current directory instead.
>
> pg_upgrade already requires a writable current directory, even without
> --initdb.
>
> So, logging to a temporary location therefore add no new requirement
> for write access to that directory.

Oh, I see. (But see previous point).

>>> --check --initdb retains the initialized new cluster whether the
>>> compatibility
>>> checks succeed or fail. After a successful check, rerun pg_upgrade
>>> against that
>
>>> cluster without --check, --initdb, or --initdb-options.
>
>> Does that mean that if you run "pg_upgrade .. --check --initdb" twice,
>> it will fail the second time because the new cluster already exists?
>
>> That doesn't sound nice.
>
> Right, repeated checks now reuse the existing initialized cluster rather
> than failing.

That's surprising too. I would expect "--check" to leave the directories
unmodified. Except for the logs.

Putting all this together, I think this would work (in non-check mode):

1. Check that "<new dir>" is empty or non-existent, like initdb does.
(An existing "<new dir>/pg_upgrade_output.d" dir is acceptable though)
2. Create "<new dir>/pg_upgrade_output.d" and the log files
3. Run "initdb <new dir>/temporary_data_dir"
4. Move all contents of "<new dir>/temporary_data_dir" to "<new dir>"
5. Delete the now-empty "<new dir>/temporary_data_dir" directory.

In --check mode, you can go through the same motions, except that in
step 4 you can just delete "temporary_data_dir".

One detail that's not quite right with this is the --allow-group-access
again. With the above, the "<new dir>" would not have the right
permissions, because pg_upgrade doesn't know the right permissions in
step 1 yet. Maybe you could fix that up around steps 4-5: read the
permissions of "<new dir>/temporary_data_dir" and copy them to "<new dir>".

The other approach I mentioned above is to teach initdb to ignore
"pg_upgrade_output.d". Then the sequence would be simply:

1. Create "<new dir>/pg_upgrade_output.d" and the log files
2. Run "initdb -D <new dir>". It ignores "pg_upgrade_output.d", but will
complain if there's anything else in "<new dir>"

In --check mode, it's a little more complicated though:

1. Create "<new dir>/pg_upgrade_output.d" and the log files
2. Run "initdb -D <new dir>". It ignores "pg_upgrade_output.d", but will
complain if there's anything else in "<new dir>"
3. Delete everything under "<new dir>" except for "pg_upgrade_output.d"

Step 3 is a bit scary. So we could still use a temporary dir in --check
mode:

1. Create "<new dir>/pg_upgrade_output.d" and the log files
2. Run "initdb -D <new dir>/temporary_data_dir".
3. Delete "<new dir>/temporary_data_dir"

The downside of that is that the --check mode diverges more from the
real mode, so there's some risk of things working in --check mode, but
failing when you run it for real.

I'm not sure which of these approaches I like the best. What do you
think? But I think the overall goals are:

- the logs always go under "<new dir>/pg_upgrade_output.d"
- the logs contain the initdb output too
- --check mode leaves the target dir unmodified

- Heikki