Skip site navigation (1) Skip section navigation (2)

Online base backup from the hot-standby

From: Jun Ishiduka <ishizuka(dot)jun(at)po(dot)ntts(dot)co(dot)jp>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Online base backup from the hot-standby
Date: 2011-05-27 06:09:30
Message-ID: 201105270609.p4R694Lo010621@ccmds32.silk.ntts.co.jp (view raw or flat)
Thread:
Lists: pgsql-hackers
Hi

I would like to develop function for 'Online base backup from the
hot-standby' in PostgreSQL 9.2.

Todo : Allow hot file system backups on standby servers
(http://wiki.postgresql.org/wiki/Todo)


[GOAL]
 * Make pg_basebackup to execute to the hot-standby server 
   and acquire online-base-backup .
     - pg_basebackup can be executed to only primary server in 
       PostgreSQL 9.1 .
     - But physical-copy(etc) under processing of pg_basebackup
       raises the load of primary server .
     - Therefore , this function is necessary .

[Problem]
(There is the following problems when hot-standby acquires 
 online-base-backup like executing pg_basebackup to the primary 
 server .)
 * pg_start_backup() and pg_stop_backup() can't be executed to the 
   hot-standby server .
     - hot-standby can't insert backup-end record to WAL-files and 
       can't operate CHECKPOINT .
        - Because hot-standby can't write anything in WAL-files .
 * hot-standby can't send WAL-files to archive server.
     - when pg_stop_backup() is executed to the primary server , 
       it waits for completing sending wal to archive server ,  
       but hot-standby can't do it.

[Policy]
(I create with the following Policy .)
     * This function doesn't affect primary server . 
        - I don't adopt the way which "hot-standby requests primary to
          execute pg_basebackup" , because I think about many standbys 
          is connected with a primary .

[Approach]
 * When pg_basebackup is executed to the hot-standby server , it 
   executes RESTARTPOINT instead of CHECKPOINT . 
   backup_label is made from the RESTARTPOINT's results , and is sent 
   to the designated backup server using pg_basebackup connection .
 * Instead of inserting backup-end record , hot-standby writes 
   backup-end-position in backup-history-file and sends to the 
   designated backup server using pg_basebackup connection .
    - In 9.1 , startup process knows backup-end-position from only 
      backup-end record . In addition to its logic, startup process 
      can know backup-end-position from backup-history-file . 
      As a result , startup process can recovery certainly 
      without backup-end record .

[Precondition]
(As a result of the above-mentioned Policy and Approach , there is 
 the following restrictions .)
     * Immediately after backup starting of WAL must contain 
       full page writes . But the above-mentioned Approach can't satisfy 
       the restriction according to circumstances . Because 
       full_page_writes of primary might equal 'off' .
       When standby recovery WAL which is removed full page writes by pg_lesslog
       , it is the same .
     * Because recovery starts from last CHECKPOINT , it becomes long .
     * I has not thought new process that become taking the place of 
       waiting for completing sending wal to archive server , yet.

[Working Step]
 STEP1: Make startup process to acquire backup-end-position from 
        not only backup-end record but also backup-history-file .
          * startup process allows to acquire backup-end-position 
            from backup-history-file .
          * When pg_basebackup is executed , backup-history-file is 
            sent to the designated backup server .
 
 STEP2: Make pg_start_backup() and pg_stop_backup() to be executed 
        by the hot-standby server.
 
[Action until The first CommitFest (on June 15)]
I will create a patch to STEP1 .
(The patch will be able to settle a problem of Omnipitr-backup-slave.)
(a problem of Omnipitr-backup-slave : 
    http://archives.postgresql.org/pgsql-hackers/2011-03/msg01490.php)
  * Shedule of creating STEP2 is the next CommitFest (in September 15) 



--------------------------------------------
Jun Ishizuka
NTT Software Corporation
TEL:045-317-7018
E-Mail: ishizuka(dot)jun(at)po(dot)ntts(dot)co(dot)jp
--------------------------------------------



Responses

pgsql-hackers by date

Next:From: Ibrar AhmedDate: 2011-05-27 08:15:13
Subject: Re: "errno" not set in case of "libm" functions (HPUX)
Previous:From: Stephen FrostDate: 2011-05-27 04:13:38
Subject: Re: Pre-alloc ListCell's optimization

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group