Error "new timeline 2 forked off current database system timeline" in cascaded replication, when master changes

From: "Abhilash Mannathanil (amannath)" <amannath(at)cisco(dot)com>
To: "pgsql-bugs(at)postgresql(dot)org" <pgsql-bugs(at)postgresql(dot)org>
Subject: Error "new timeline 2 forked off current database system timeline" in cascaded replication, when master changes
Date: 2019-02-11 05:57:24
Message-ID: BDC9E61A-CDCC-458A-8364-E739A548C2B5@cisco.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi,

Postgres Version - PostgreSQL 9.6.5 on x86_64-pc-linux-gnu, compiled by gcc (Debian 4.9.2-10) 4.9.2, 64-bit
Host - 4.15.0-43-generic #46~16.04.1-Ubuntu SMP Fri Dec 7 13:31:08 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

I am experimenting with a cascaded replication between two datacenters and below is my setup (refer to attached image if the alignment is not proper).

________________________________________
| DC-1 |
| Server-1(master) |
| /\ \ |
| / \Sync \Async |
| / \ \ |
| Server2(slave) Server3(slave) \ |
|____________________________________\__ |
\ VIP
___________________\_____________________
|DC-2 / |
| Server-4 (slave) |
| /\ |
| / \Sync |
| / \ |
|Server5(slave) Server6(slave) |
|________________________________________|

I have setup a virtual IP in DC-1 and the Server-4 in datacenter-2 is configured with upstream server as the virtual ip address. After the initial set-up everything is working fine.

Then I have stopped the server-1(master) and promoted Server-2 as master. Also configured the VIP in server 2. After this, in the logs of Server-4, I am seeing the below error:

2019-02-10 21:00:25.708 PST [27602] LOG: restarted WAL streaming at F/EC000000 on timeline 1
2019-02-10 21:00:25.840 PST [27602] LOG: replication terminated by primary server
2019-02-10 21:00:25.840 PST [27602] DETAIL: End of WAL reached on timeline 1 at F/EC7DD488.
2019-02-10 21:00:25.841 PST [14973] LOG: new timeline 2 forked off current database system timeline 1 before current recovery point F/EC7DD4C0

Server-4 did not recover from this error.

After restarting the server-4, the below error is displayed:

2019-02-10 21:01:10.131 PST [23920] FATAL: requested timeline 2 does not contain minimum recovery point F/EC7DD4C0 on timeline 1
2019-02-10 21:01:10.133 PST [23915] LOG: startup process (PID 23920) exited with exit code 1
2019-02-10 21:01:10.133 PST [23915] LOG: aborting startup due to startup process failure
2019-02-10 21:01:10.136 PST [23915] LOG: database system is shut down

Any pointers appreciated.
regards
Abhilash Mannathanil

Attachment Content-Type Size
image/png 20.0 KB

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message David Rowley 2019-02-11 08:25:24 Re: BUG #15629: Typo in Documentation
Previous Message PG Bug reporting form 2019-02-11 03:26:13 BUG #15629: Typo in Documentation