Re: Sync Rep: First Thoughts on Code

Hi,

On Wed, Dec 24, 2008 at 2:37 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>
> On Wed, 2008-12-24 at 02:23 +0900, Fujii Masao wrote:
>
>> Oh, sorry. I don't want to scare you ;) But, yes, it's important. We should
>> rethink the question? "Why does the failed server always need a fresh
>> backup?" Though we discussed it previously and concluded that it should
>> be done next time.
>> http://archives.postgresql.org/pgsql-hackers/2008-11/msg01612.php
>
> We might ask why pg_start_backup() needs to perform checkpoint though,
> since you have remarked that is a problem also.
>
> The answer is that it doesn't really need to, we just need to be certain
> that archiving has been running since whenever we choose as the start
> time. So we could easily just use the last normal checkpoint time, as
> long as we had some way of tracking the archiving.
>
> ISTM we can solve the checkpoint problem more easily and it would
> potentially save much more time than "tuning rsync for Postgres", which
> is what the other idea amounted to. So I do see a solution that is both
> better and more quickly achievable for 8.4.

Sounds good. I agree that pg_start_backup basically doesn't need
checkpoint. But, for full_page_write == off, we probably cannot get
rid of it. Even if full_page_write == on, since we cannot make out
whether all indispensable full pages were written after last checkpoint,
pg_start_backup must do checkpoint with "forcePageWrite = on".

Problem is that online backup itself is unsafe. Even if there is no
disk failure (i.e. normal case), we can easily produce a partial write
in online backup. So, we always need full pages when recovering
online backup, then pg_start_backup always needs checkpoint
with forcePageWrite = on.

I think that we probably have to track the history of full_page_write,
in order to get rid of checkpoint from pg_start_backup.

On the other hand, the data after crash other than media crash
is "safe". Currently, we can recover it without full page write
as simple crash recovery case. I think that we can use it also for
archive recovery, because there isn't really any distinction between
both. I've not found the corner case yet. Do you have?

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center