Re: [HACKERS] Point in Time Recovery

From: Simon Riggs <simon(at)2ndquadrant(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>, Mark Kirkwood <markir(at)coretech(dot)co(dot)nz>, pgsql-admin(at)postgresql(dot)org
Subject: Re: [HACKERS] Point in Time Recovery
Date: 2004-07-22 07:43:18
Message-ID: 1090482198.2660.4.camel@localhost.localdomain
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin pgsql-hackers pgsql-patches

On Thu, 2004-07-22 at 04:29, Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I think we should push the partially complete WAL file to the archive
> > location before shutdown. ...
> > When you are running and finally fill up the WAL file it would then
> > overwrite the one in the archive but I think that is OK.
>
> I don't think this can fly at all. Here are some off-the-top-of-the-head
> objections:
>
> 1. We don't have the luxury of spending indefinite amounts of time to
> do a database shutdown. Commonly we are under a twenty-second sentence
> of death from init. I don't want to spend the 20 seconds waiting to see
> if the archiver will manage to push 16MB onto a slow tape drive. Also,
> if the archiver does fail to push the data in time, it'll likely leave a
> broken (partial) xlog file in the archive, which would be really bad
> news if the user then relies on that.
>
> 2. What if the archiver process entirely fails to push the file? (Maybe
> there's not enough disk space, for instance.) In normal operation we'll
> just retry every so often. We definitely can't do that during shutdown.
>
> 3. You're blithely assuming that the archival process can easily provide
> overwrite semantics for multiple pushes of the same xlog filename. Stop
> thinking about "cp to some directory" and start thinking "dump to tape"
> or "burn onto CD" or something like that. We'll be raising the ante
> considerably if we require the archive_command to deal with this.
>
> I think the last one is really the most significant issue. We have to
> keep the archiver API as simple as possible.
>

Not read whole chain of conversation...but this idea came up before and
was rejected then. I agree with the 3 objections to that thought above.

There's already enough copies of full xlogs around to worry about.

If you need more granularity, reduce size of xlog files....

(Tom, SUID would be the correct timeline id in that situation? )

More later, Simon Riggs

In response to

Browse pgsql-admin by date

  From Date Subject
Next Message Eduardo S. Fontanetti 2004-07-22 15:03:03 pg_dumpall with lo
Previous Message Bruce Momjian 2004-07-22 03:54:48 Re: [HACKERS] Point in Time Recovery

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2004-07-22 07:46:59 Re: PreallocXlogFiles
Previous Message Christopher Kings-Lynne 2004-07-22 07:37:20 Re: Fixing PKs and Uniques in tablespaces

Browse pgsql-patches by date

  From Date Subject
Next Message Andreas Pflug 2004-07-22 08:16:42 Re: logfile subprocess and Fancy File Functions
Previous Message Peter Eisentraut 2004-07-22 05:11:02 Re: win32 readline