26.3. Record-based Log Shipping for High Availability

A record-based log shipping is a built-in replication solution in PostgreSQL 8.4 onwards. WAL records are transferred from the primary server to the standby server via dedicated processes for log shipping, then replayed continuously by the standby. This transfer and recovery are performed per records, so they can be performed before WAL file fills. Unlike file-based log shipping, archive_timeout doesn't need to be configured.

This built-in replication can synchronize log shipping with a transaction processing, ie you can control how long transaction commit waits for log shipping before the commit command returns a success to the client. Log shipping with this capability is also referred to as synchronous replication. It is guaranteed that the transaction is already available in both servers when the client recognizes that it has been committed. So, even if the primary server fails, the standby server can take over the database processing safely (ie, without any transaction loss).

26.3.1. Setup of the Primary Server

The short procedure for configuring a primary server is as follows. For full details of each step, refer to previous sections as noted.

  1. Set up continuous archiving from the primary to a WAL archive located in a directory which is accessible from the standby server. Ensure that archive_mode and archive_command are set appropriately on the primary (see Section 24.3.1). This is because WAL files generated in the primary server before this built-in replication starts have to be transferred to the standby server by using file-based log shipping. When archive_mode is unsent, only WAL files which are not sent to the standby server can be archived, so you can prevent WAL files from being transferred doubly by file-based and record-based log shipping.

  2. Set up connections and authentication so that the standby server can successfully connect to the replication database of the primary server (see Section 19.3).

    On systems that support the keepalive socket option, setting tcp_keepalives_idle, tcp_keepalives_interval and tcp_keepalives_count helps you to find the troubles with replication (eg. the network outage or the failure of the standby server) as soon as possible. Note that transaction processings on the primary server might be blocked until the trouble with replication is found when it occurs.

  3. Start the PostgreSQL server on the primary to commence normal database operations.

26.3.2. Setup of the Standby Server

The short procedure for configuring a standby server is as follows. For full details of each step, refer to previous sections as noted.

  1. Make a base backup of the primary server (see Section 24.3.2), and load this data onto the standby. Note that all files present in pg_xlog and pg_xlog/archive_status on the standby server have to be removed because they might be obsolete.

  2. Set up continuous archiving, connections and authentication like the primary server, because the standby server might work as a primary server after failover. Ensure that your settings are consistent with the future environment after the primary and the standby server are interchanged by failover.

  3. Create a recovery command file recovery.conf that specifies a restore_command that waits as described previously (see Section 24.3.3). The built-in replication requires also the following parameters.

    enable_replication (boolean)

    Specifies whether to enable the built-in replication. When this parameter is on, the PostgreSQL server works as a standby and tries to receive the WAL records from the primary server continuously. The default is off, which allows only an archive recovery without replication. So, the built-in replication requires this parameter to be explicitly set to on.

    replication_primary_host (string)

    Specifies host or hostaddr which the primary server listens for the connection from the standby server. Ensure that the same value is set to also listen_addresses on the primary server. If this parameter is omitted, PGHOST or PGHOSTADDR is used instead.

    replication_primary_port (integer)

    Specifies port which the primary server listens on. If this parameter is omitted, PGPORT is used instead.

    replication_user (string)

    Specifies user with the access privilege to the replication database on the primary server. If this parameter is omitted, PGUSER is used instead.

    recovery_trigger_file is also useful to control whether to promote the standby server to the primary after replication is terminated. If a trigger file is specified, the standby server doesn't finish recovery and works as a primary until the file is found. This is useful for preventing a split-brain syndrome (ie. two or more primary servers are still in progress, there is no consistency of the transactions between them). You can leave the determination of promotion to outside (eg. clusterware with quorum server) of PostgreSQL. Note that replication is terminated as a side-effect if the trigger file is found while replication is in progress. Even in this case, recovery is finished after all the available WAL records are applied. If this parameter is omitted, termination of replication automatically causes recovery to end whether you want or not.

  4. Set up ~/.pgpass (see Section 31.15) and the environment variables (see Section 31.14) so that the standby server can successfully connect to the replication database of the primary server (see Section 19.3), if needed.

  5. Start the PostgreSQL server on the standby to receive the WAL records from the primary server and apply them continuously.