Re: Why we really need timelines *now* in PITR

On Mon, 2004-07-19 at 04:31, Tom Lane wrote:
> Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> > The way you write this makes me think you might mean you would allow: we
> > can start recovering in one timelines, then rollforward takes us through
> > all the timeline nexus points required to get us to the target
> > timeline.
>
> Sure. Let's draw a diagram:
>
> 0001.0014 - 0001.0015 - 0001.0016 - 0001.0017 - ...
> |
> + 0002.0016 - 0002.0017 - ...
> |
> + 0003.0017 - ...
>
> If you decide you would like to recover to someplace in timeline 0002,
> you need to take the 0002 log files where they exist, and the 0001
> log files where there is no 0002, except you do not revert to 0001
> once you have used an 0002 file (this restriction is needed in case
> the 0001 timeline goes to higher segment numbers than 0002 has reached).
> In no case do you use an 0003 file.
>
> > I had imagined that recovery would only ever be allowed to start and end
> > on the same timeline. I think you probably mean that?
>
> Logically it's all one timeline, I suppose, but to implement it
> physically that way would mean duplicating all past 0001 segments when
> we want to create the 0002 timeline. That's not practical and not
> necessary.
>
> > Another of the issues I was thinking through was what happens at the end
> > of your scenario abobe
> > - You're on timeline 1 and you need to perform recovery.
> > - You perform recovery and timeline 2 is created.
> > - You discover another error and decide to recover again.
> > - You recover timeline 1 again: what do you name the new timeline
> > created? 2 or 3?
>
> You really want to call it 3. To enforce this mechanically would
> require having a counter that sits outside the $PGDATA directory and
> is not subject to being reverted by a restore-from-backup. I don't
> see any very clean way to do that at the moment --- any thoughts?
>
> In the absence of such a counter we could ask the DBA to specify a new
> timeline number in recovery.conf, but this strikes me as one of those
> easy-to-get-wrong things ...
>
> One possibility is to extend the archiving API so that we can inquire
> about the largest timeline number that exists anywhere in the archive.
> If we take new timeline number = 1 + max(any in archive, any in pg_xlog)
> then we are safe. But I'm not really convinced that such a thing would
> be any less error-prone than the manual way :-(, because for any
> archival method that's more complicated than "cp them all into one
> directory", it'd be hard to extract the max stored filename.
>

Think the same as you do on all of that. Excellent.

Some further thinking from that base...

Perhaps timelines should be nest-numbered: (using 0 as a counter also)
0 etc is the original branch
0.1 is the first recovery off the original branch
0.2 is the second recovery off the original branch
0.1.1 is the first recovery off the first recovery (so to speak)
0.1.2 is the second etc
That way you don't have the problem of "which is 3?" in the examples
above. [Would we number a recovery of 1 as 3 or would then next recovery
off 2 be numbered 3?]

Not necessarily the way we would show that as a timeline number. It
could still be shown as a single hex number representing each nesting
level as 4 bits...(restricting us to 7 recoveries per timeline...)

Just as a thought, DB2 uses the concept of a history file also....

If we go with the renaming recovery.conf when it completes, why not make
that the record of previous recoveries? Move it to archive_status and
name it according to the timeline it just created, e.g.
recovery.done.<timeline>.<timestamp>

If you're re-restoring into the same directory you wouldn't then
overwrite the history of previous recoveries.

It would be an extremely bad thing to have to specify the timeline
number, and I agree we can't ask the archive.

Best Regards, Simon Riggs