Re: Patch for migration of the pg_commit_ts directory

Hello

Without the origin preservation patch Sawada-san's scenario upthread
still reproduces:

-- On the old subscriber (track_commit_timestamp=on):
CREATE TABLE test (id int primary key, t text);
CREATE SUBSCRIPTION mysub CONNECTION '...' PUBLICATION pub;
CREATE SUBSCRIPTION a_dummy CONNECTION '...' PUBLICATION pub WITH
(connect = false, enabled = false);
-- wait for mysub's table sync to reach 'r', then on the publisher:
INSERT INTO test VALUES (1, 'pub-insert');
-- subscriber: pg_xact_commit_timestamp_origin(xmin).roident = 1 (mysub)
ALTER SUBSCRIPTION mysub DISABLE;

Then pg_upgrade --preserve-commit-timestamps into a new cluster with
track_commit_timestamp=on, enable mysub, and run UPDATE test SET t =
'pub-update' WHERE id = 1 on the publisher. Origins are recreated in
subscription name order, so a_dummy gets roident 1 and mysub gets 2,
but the copied commit_ts entry still says 1:

LOG: conflict detected on relation "public.test":
conflict=update_origin_differs
DETAIL: Updating the row that was modified by a different origin
"pg_16408" in transaction 672 ...: local row (1, pub-insert), remote
row (1, pub-update), replica identity (id)=(1).
CONTEXT: processing remote data for replication origin "pg_16409"
during message type "UPDATE" ...

Even as an opt-in feature, this doesn't seem right if this gets
committed first. Maybe that has to be an explicit prerequisite for the
patch, or until the origin preservation patch gets committed,
pg_upgrade could refuse or at least warn when the option is used and
the old cluster has subscriptions?

And there's another corner case issue: the commit_ts window is read
before the old server is started, and it isn't re-read later.

If that start runs with track_commit_timestamp=off,
DeactivateCommitTs() deletes everything in pg_commit_ts. pg_upgrade
then copies the empty directory, sets the stale window with
pg_resetwal -c, and finishes successfully. That cam be reproduced in
multiple ways: either the old cluster's postgresql.conf was changed to
off before the shutdown for the upgrade, or pg_upgrade -o "-c
track_commit_timestamp=off".

After the upgrade, once the old xids are more than one segment behind nextXid:

SELECT pg_xact_commit_timestamp('666'::xid);
ERROR: could not open file "pg_commit_ts/0000": No such file or directory
DETAIL: Could not access commit timestamp of transaction 666.

- Commit timestamps and origin data are not preserved during the upgrade.
- As a result, even if
+ FIXME: must be updated after the replication origin can be preserved.
+ Origin data is not preserved during the upgrade. As a result, even if

Shouldn't this also mention that preserve-commit-timestamps is opt-in?