Re: pg_rewind does not rewind diverging timelines

From: Japin Li <japinli(at)hotmail(dot)com>
To: Mats Kindahl <mats(dot)kindahl(at)gmail(dot)com>
Cc: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, pgsql-hackers mailing list <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: pg_rewind does not rewind diverging timelines
Date: 2026-07-17 15:59:49
Message-ID: SY7PR01MB10921220B1B7054260BC8031FB6C62@SY7PR01MB10921.ausprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Hi, all

On Sun, 21 Jun 2026 at 11:09, Mats Kindahl <mats(dot)kindahl(at)gmail(dot)com> wrote:
> On 6/8/26 12:48, Andrey Borodin wrote:
>
> On 30 Apr 2026, at 13:19, Mats Kindahl <mats(dot)kindahl(at)gmail(dot)com> wrote:
>
> There is one scenario that I assume is known that TLC found, but does not seem to be fixed. It is a relatively rare case, but since the fix is quite easy, I thought I'd share it with you and get feedback.
>
> Hi Mats,
>
> Hi Andrey,
>
> Thanks for looking at this.
>
> Thanks for working on this. I think the problem is real, but I wonder if
> adding a separate UUID to timeline history files is solving it one step
> too late.
>
> If two independent promotions manage to choose the same numeric TLI, then
> we already have two different histories with the same timeline identifier.
> Their history files will also have the same name. A UUID in the file lets
> tools detect the mismatch afterwards, but it does not prevent the archive
> namespace from containing two different meanings for the same TLI.
>
> Yes, that is correct.
>
> In normal deployments with a shared archive this should only be possible
> when the history file is not visible to the other promoting server:
> either there is no usable restore_command/shared archive, or there is a
> race around publishing and observing the history file. In other words, TLI
> allocation is not atomic, but it is intended to be coordinated through the
> archive.
>
> Yes, that is the ideal way it should work when you have a shared archive. This works because you have a central authority
> that synchronizes the timelines (in theory, not counting bugs).
>
> Maybe we should keep TimelineID as the actual branch identifier and make
> that allocation harder to collide instead of adding a second identifier.
> For example, when choosing a new TLI, add some randomness rather than just
> using the next sequential value.
>
> That would make the race window much less
> dangerous: two independent promotions would be extremely unlikely to
> choose the same TLI, the history file names would remain distinct, and TLI
> would keep its current role as the timeline identifier.
> This also keeps the operational model simpler. TimelineID is already the
> identifier exposed in WAL file names, history file names, logs, and
> recovery configuration. If we add UUIDs, we effectively introduce another
> identity for the same object, and tools then need to reason about both.
> If instead we make TLI allocation less deterministic under races, the
> existing model remains intact.
>
> Does that framing make sense, or am I missing a case where duplicate TLIs
> are unavoidable even with a shared archive and a less collision-prone
> allocation scheme?
>
> I considered using some random increment of the TLI in the manner you describe but there are some issues that makes this
> solution more complicated from an operational perspective:
>
> * If you skip some TLIs (in the sense pick a TLI that is "random but larger"), then it is not clear what the relation
> between them are.
>
> * The history files contain the complete linkage of the timelines, so that is covered, but the naming would be strange.
>
> * For example, if you have history files 1, 5, 7, and 8, then these can all belong to different timelines, (except 1), or
> be a single timeline and it is hard to understand which one without looking through the files.
>
> * With more promotions, the relation becomes even more strange, and the risk of collisions increases. (For example,
> imagine one timeline with 1, 5, 7, 8, 11, and one timeline that forks off 1. Then any increment of 4, 6, 7, or 10 will
> result in a collision.)
>
> * To actually reduce the risk significantly, you need to have a very wide range of the added randomness. Taking a smaller
> number is easier to work with, but then you need to handle that some timelines can collide in some manner.
> * Normally, the history file with the highest number will be the only relevant one. With this approach, you have to check
> the contents of the files to understand which ones are relevant, which increases the operational burden.
>
> In contrast, if you use an UUID in this manner.
>
> * Adding an UUID does not require a central coordinator and is not likely to collide (on the level "impossible to
> collide") and is very straightforward to add. It also comes with a low risk since the places in the code that requires
> changes are very few and not likely to have unexpected consequences elsewhere. This works both with and without a
> shared archive.
> * Normally, a shared archive should only contain a single timeline. Anything else is an anomaly and should be corrected.
> * I think it is still necessary to handle the case where you do not have a shared archive; it would be an odd limitation
> to say that promote only works if you have a shared archive
> * The UUID still serves a purpose in capturing a situation where things have gone wrong. Think of the UUID as similar to
> a "checksum" safety and an extra precaution to prevent things from going wrong.
>
> In short, I think the operational issues with random increment of the history file number is worse, not better, and we
> should deal with the name collisions correctly for shared archives instead. There is an issue in that it need to work
> even in the case where you have a promotion that generates a new UUID but the correct history file exists (reported in
> the other message) that I will look into.
>

I would like to know the current status of this patch. I have encountered the
same issue in practice, and I think the proposed solution is reasonable.

I found that the v6 patch does not apply cleanly to the current master (1f414035135)
because commit 7f77b2a89bd4 changed the parameter type of writeTimeLineHistory().

I've rebased the patch and attached v7.

> Best wishes,
> Mats Kindahl
>
> Best regards, Andrey Borodin.

--
Regards,
Japin Li
ChengDu WenWu Information Technology Co., Ltd.

Attachment Content-Type Size
v7-0001-pg_rewind-use-UUIDs-to-detect-independent-same-TL.patch text/x-patch 37.8 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2026-07-17 16:02:21 Re: Heads Up: cirrus-ci is shutting down June 1st
Previous Message Jelte Fennema-Nio 2026-07-17 15:55:30 Re: ci: namespace ccache by PostgreSQL major version