Re: Why we really need timelines *now* in PITR

Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> On Mon, 2004-07-19 at 19:33, Tom Lane wrote:
>> When doing WAL archiving a history file can be copied
>> off to the archive area by the existing archiver mechanism (ie, we'll
>> make a .ready file for it as soon as it's written).

> Need to check the archive code which relies on file shape and length

Yeah. I made some adjustments already with the expectation that we'd
want to do this, but it'll take a little bit more weakening of the
code's tests.

>> * When we need to do recovery, we first identify the source timeline
>> (either by reading the current timeline ID from pg_control, or the DBA
>> can tell us with a parameter in recovery.conf).

> ** Surely it is the backup itself that determines the source timeline?

The backup determines the starting point, but there may be several
timelines you could follow after that (especially in the scenario where
you're redoing a recovery starting from the same backup). The point
here is that there could be timeline branches after the backup
occurred. So yes the backup has to be in an ancestral timeline, but not
necessarily exactly the recovery-target timeline.

> ...thinking....recovery.conf would need to specify:
> recovery_target (if there is one, either a time or txnid)
> recovery_target_timeline (if there is one, otherwise end of last one)
> recovery_target_history_file (which specifies how the timeline ids are
> sequenced)

No, the source timeline is not necessarily associated with a
recovery_target --- for instance you might want it to run to the end of
a particular timeline. I suspect it might be more confusing than
helpful to use the term "target timeline".

>> try to open it with the source timeline ID; if that doesn't exist, try
>> the immediate parent timeline ID; then the grandparent, etc.

> This jigging around is OK, because most people will be using only one
> timeline anyhow, so its not likely to cause too much fuss for the user.

It might confuse someone who's watching the sequence of archive
retrieval requests, but as long as that's all mechanized it doesn't
seem like there's any real potential for trouble.

> ** I would prefer to add a random number to the timeline as a way of
> identifying the next one. This will produce fewer probes, so less wasted
> tape mounts,

How do you figure that? Seems like the same number of probes either way.

> but most importantly it gets round this issue:

> You're on timeline X, then you recover and run for a while on timeline
> Y. You then realise recovering to that target was a really bad idea for
> some reason (some VIPs record wasn't in the recovered data etc). We then
> need to re-recover from the backup on X to a new timeline, Z. But how
> does X know that Y existed when it creates Z?

Because there is a Y.history file laying about (either in the archive or
pg_xlog).

We will need to recommend to DBAs that they not delete Y.history from
the archive unless they've already deleted all Y.whatever log segments.
Once they have done this, the past existence of timeline Y is no longer
of interest and so there'd be no real problem in recycling the ID.
I would say the above is just as true if you use random IDs as if you
use sequential ones. I distrust systems that assume there will never be
a collision of "randomly-chosen" IDs.

> If Y = f(x) in a deterministic way, then Y will always == Z. Of course,
> we could provide an id, but what would you pick? The best way is to get
> out of trouble by picking a new timeline id that's very unlikely to have
> been picked before.

I do not see the advantage.

regards, tom lane