From 3d5ba741331839d58e7b5b7db3e189f119cdccbe Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Tue, 20 Nov 2018 11:33:31 +0100 Subject: [PATCH v7] Integrate recovery.conf into postgresql.conf recovery.conf settings are now set in postgresql.conf (or other GUC sources). Currently, all the affected settings are PGC_POSTMASTER; this could be refined in the future case by case. Recovery is now initiated by a file recovery.signal. Standby mode is initiated by a file standby.signal. The standby_mode setting is gone. If a recovery.conf file is found, an error is issued. The trigger_file setting has been renamed to promote_trigger_file as part of the move. The documentation chapter "Recovery Configuration" has been integrated into "Server Configuration". pg_basebackup -R now appends settings to postgresql.auto.conf and creates a standby.signal file. Author: Simon Riggs Author: Abhijit Menon-Sen Author: Sergei Kornilov Discussion: https://www.postgresql.org/message-id/flat/607741529606767@web3g.yandex.ru/ --- contrib/pg_standby/pg_standby.c | 2 +- doc/src/sgml/backup.sgml | 22 +- doc/src/sgml/config.sgml | 515 +++++++++++++++++- doc/src/sgml/filelist.sgml | 1 - doc/src/sgml/func.sgml | 2 +- doc/src/sgml/high-availability.sgml | 60 +- doc/src/sgml/pgstandby.sgml | 2 +- doc/src/sgml/postgres.sgml | 1 - doc/src/sgml/recovery-config.sgml | 510 ----------------- doc/src/sgml/ref/pg_basebackup.sgml | 7 +- doc/src/sgml/ref/pg_rewind.sgml | 8 +- doc/src/sgml/ref/pgarchivecleanup.sgml | 4 +- doc/src/sgml/ref/pgupgrade.sgml | 2 +- doc/src/sgml/release-10.sgml | 2 +- doc/src/sgml/release-9.1.sgml | 5 +- doc/src/sgml/release-9.4.sgml | 15 +- doc/src/sgml/release-9.5.sgml | 8 +- doc/src/sgml/release.sgml | 3 +- src/backend/Makefile | 4 +- .../access/transam/recovery.conf.sample | 158 ------ src/backend/access/transam/xlog.c | 500 ++++++----------- src/backend/access/transam/xlogarchive.c | 4 +- src/backend/commands/extension.c | 4 +- src/backend/utils/misc/guc.c | 421 ++++++++++++++ src/backend/utils/misc/postgresql.conf.sample | 47 ++ src/bin/pg_archivecleanup/pg_archivecleanup.c | 2 +- src/bin/pg_basebackup/pg_basebackup.c | 148 +++-- src/bin/pg_basebackup/t/010_pg_basebackup.pl | 15 +- src/bin/pg_rewind/RewindTest.pm | 10 +- src/include/access/xlog.h | 39 +- src/include/utils/guc_tables.h | 2 + src/port/quotes.c | 2 +- src/test/perl/PostgresNode.pm | 26 +- src/test/recovery/t/001_stream_rep.pl | 4 +- src/test/recovery/t/003_recovery_targets.pl | 2 +- src/test/recovery/t/004_timeline_switch.pl | 4 +- src/test/recovery/t/005_replay_delay.pl | 2 +- src/test/recovery/t/009_twophase.pl | 6 +- .../t/010_logical_decoding_timelines.pl | 2 +- src/test/recovery/t/012_subtransactions.pl | 6 +- 40 files changed, 1404 insertions(+), 1173 deletions(-) delete mode 100644 doc/src/sgml/recovery-config.sgml delete mode 100644 src/backend/access/transam/recovery.conf.sample diff --git a/contrib/pg_standby/pg_standby.c b/contrib/pg_standby/pg_standby.c index ee1fbd7b33..946239c2ac 100644 --- a/contrib/pg_standby/pg_standby.c +++ b/contrib/pg_standby/pg_standby.c @@ -611,7 +611,7 @@ usage(void) printf(" -w MAXWAITTIME max seconds to wait for a file (0=no limit) (default=0)\n"); printf(" -?, --help show this help, then exit\n"); printf("\n" - "Main intended use as restore_command in recovery.conf:\n" + "Main intended use as restore_command in postgresql.conf:\n" " restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n" "e.g.\n" " restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n"); diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index 3fa5efdd78..a73fd4d044 100644 --- a/doc/src/sgml/backup.sgml +++ b/doc/src/sgml/backup.sgml @@ -1220,8 +1220,11 @@ Recovering Using a Continuous Archive Backup - Create a recovery command file recovery.conf in the cluster - data directory (see ). You might + Set recovery configuration settings in + postgresql.conf (see ) and create a file + recovery.signal in the cluster + data directory. You might also want to temporarily modify pg_hba.conf to prevent ordinary users from connecting until you are sure the recovery was successful. @@ -1232,8 +1235,8 @@ Recovering Using a Continuous Archive Backup proceed to read through the archived WAL files it needs. Should the recovery be terminated because of an external error, the server can simply be restarted and it will continue recovery. Upon completion - of the recovery process, the server will rename - recovery.conf to recovery.done (to prevent + of the recovery process, the server will remove + recovery.signal (to prevent accidentally re-entering recovery mode later) and then commence normal database operations. @@ -1249,12 +1252,9 @@ Recovering Using a Continuous Archive Backup - The key part of all this is to set up a recovery configuration file that + The key part of all this is to set up a recovery configuration that describes how you want to recover and how far the recovery should - run. You can use recovery.conf.sample (normally - located in the installation's share/ directory) as a - prototype. The one thing that you absolutely must specify in - recovery.conf is the restore_command, + run. The one thing that you absolutely must specify is the restore_command, which tells PostgreSQL how to retrieve archived WAL file segments. Like the archive_command, this is a shell command string. It can contain %f, which is @@ -1316,7 +1316,7 @@ Recovering Using a Continuous Archive Backup If you want to recover to some previous point in time (say, right before the junior DBA dropped your main transaction table), just specify the - required stopping point in recovery.conf. You can specify + required stopping point. You can specify the stop point, known as the recovery target, either by date/time, named restore point or by completion of a specific transaction ID. As of this writing only the date/time and named restore point options @@ -1414,7 +1414,7 @@ Timelines that was current when the base backup was taken. If you wish to recover into some child timeline (that is, you want to return to some state that was itself generated after a recovery attempt), you need to specify the - target timeline ID in recovery.conf. You cannot recover into + target timeline ID in . You cannot recover into timelines that branched off earlier than the base backup. diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index c4effa034c..c171bfd64d 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -3000,6 +3000,377 @@ Archiving + + + Archive Recovery + + + configuration + of recovery + of a standby server + + + + This section describes the settings that apply only for the duration of + the recovery. They must be reset for any subsequent recovery you wish to + perform. They can only be set at server start and cannot be changed once + recovery has begun. + + + + Recovery covers using the server as a standby or for + executing a targeted recovery. Typically, standby mode would be used to + provide high availability and/or read scalability, whereas a targeted + recovery is used to recover from data loss. + + + + To start the server in standby mode create file called + standby.signalstandby.signal + in the data directory. The server will enter recovery and will not stop + recovery when the end of archived WAL is reached, but will keep trying to + continue recovery by connecting to the sending server as specified by the + primary_conninfo setting and/or by fetching new WAL + segments using restore_command. In this mode, you may + use parameters in both and sections. Parameters from + will not be used. + + + + To start the server in targeted recovery create a file called + recovery.signalrecovery.signal + in the data directory. If both standby.signal and + recovery.signal files are created, standby mode + takes precedence. Targeted recovery mode will end when end of archived + WAL is reached, or when recovery_target is reached. + In this mode you may use parameters from both and sections. Parameters from will not be used. + + + + + restore_command (string) + + restore_command configuration parameter + + + + + The local shell command to execute to retrieve an archived segment of + the WAL file series. This parameter is required for archive recovery, + but optional for streaming replication. + Any %f in the string is + replaced by the name of the file to retrieve from the archive, + and any %p is replaced by the copy destination path name + on the server. + (The path name is relative to the current working directory, + i.e., the cluster's data directory.) + Any %r is replaced by the name of the file containing the + last valid restart point. That is the earliest file that must be kept + to allow a restore to be restartable, so this information can be used + to truncate the archive to just the minimum required to support + restarting from the current restore. %r is typically only + used by warm-standby configurations + (see ). + Write %% to embed an actual % character. + + + + It is important for the command to return a zero exit status + only if it succeeds. The command will be asked for file + names that are not present in the archive; it must return nonzero + when so asked. Examples: + +restore_command = 'cp /mnt/server/archivedir/%f "%p"' +restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows + + An exception is that if the command was terminated by a signal (other + than SIGTERM, which is used as part of a + database server shutdown) or an error by the shell (such as command + not found), then recovery will abort and the server will not start up. + + + + + + archive_cleanup_command (string) + + archive_cleanup_command configuration parameter + + + + + This optional parameter specifies a shell command that will be executed + at every restartpoint. The purpose of + archive_cleanup_command is to provide a mechanism for + cleaning up old archived WAL files that are no longer needed by the + standby server. + Any %r is replaced by the name of the file containing the + last valid restart point. + That is the earliest file that must be kept to allow a + restore to be restartable, and so all files earlier than %r + may be safely removed. + This information can be used to truncate the archive to just the + minimum required to support restart from the current restore. + The module + is often used in archive_cleanup_command for + single-standby configurations, for example: +archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r' + Note however that if multiple standby servers are restoring from the + same archive directory, you will need to ensure that you do not delete + WAL files until they are no longer needed by any of the servers. + archive_cleanup_command would typically be used in a + warm-standby configuration (see ). + Write %% to embed an actual % character in the + command. + + + If the command returns a nonzero exit status then a warning log + message will be written. An exception is that if the command was + terminated by a signal or an error by the shell (such as command not + found), a fatal error will be raised. + + + + + + recovery_end_command (string) + + recovery_end_command configuration parameter + + + + + This parameter specifies a shell command that will be executed once only + at the end of recovery. This parameter is optional. The purpose of the + recovery_end_command is to provide a mechanism for cleanup + following replication or recovery. + Any %r is replaced by the name of the file containing the + last valid restart point, like in . + + + If the command returns a nonzero exit status then a warning log + message will be written and the database will proceed to start up + anyway. An exception is that if the command was terminated by a + signal or an error by the shell (such as command not found), the + database will not proceed with startup. + + + + + + + + + + + Recovery Target + + + By default, recovery will recover to the end of the WAL log. The + following parameters can be used to specify an earlier stopping point. + At most one of recovery_target, + recovery_target_lsn, recovery_target_name, + recovery_target_time, or recovery_target_xid + can be used; if more than one of these is specified in the configuration + file, the last entry will be used. + These parameters can only be set at server start. + + + + + recovery_target = 'immediate' + + recovery_target configuration parameter + + + + + This parameter specifies that recovery should end as soon as a + consistent state is reached, i.e. as early as possible. When restoring + from an online backup, this means the point where taking the backup + ended. + + + Technically, this is a string parameter, but 'immediate' + is currently the only allowed value. + + + + + + recovery_target_name (string) + + recovery_target_name configuration parameter + + + + + This parameter specifies the named restore point (created with + pg_create_restore_point()) to which recovery will proceed. + + + + + + recovery_target_time (timestamp) + + recovery_target_time configuration parameter + + + + + This parameter specifies the time stamp up to which recovery + will proceed. + The precise stopping point is also influenced by + . + + + + + + recovery_target_xid (string) + + recovery_target_xid configuration parameter + + + + + This parameter specifies the transaction ID up to which recovery + will proceed. Keep in mind + that while transaction IDs are assigned sequentially at transaction + start, transactions can complete in a different numeric order. + The transactions that will be recovered are those that committed + before (and optionally including) the specified one. + The precise stopping point is also influenced by + . + + + + + + recovery_target_lsn (pg_lsn) + + recovery_target_lsn configuration parameter + + + + + This parameter specifies the LSN of the write-ahead log location up + to which recovery will proceed. The precise stopping point is also + influenced by . This + parameter is parsed using the system data type + pg_lsn. + + + + + + + The following options further specify the recovery target, and affect + what happens when the target is reached: + + + + + recovery_target_inclusive (boolean) + + recovery_target_inclusive configuration parameter + + + + + Specifies whether to stop just after the specified recovery target + (true), or just before the recovery target + (false). + Applies when , + , or + is specified. + This setting controls whether transactions + having exactly the target WAL location (LSN), commit time, or transaction ID, respectively, will + be included in the recovery. Default is true. + + + + + + recovery_target_timeline (string) + + recovery_target_timeline configuration parameter + + + + + Specifies recovering into a particular timeline. The default is + to recover along the same timeline that was current when the + base backup was taken. Setting this to latest recovers + to the latest timeline found in the archive, which is useful in + a standby server. Other than that you only need to set this parameter + in complex re-recovery situations, where you need to return to + a state that itself was reached after a point-in-time recovery. + See for discussion. + + + + + + recovery_target_action (enum) + + recovery_target_action configuration parameter + + + + + Specifies what action the server should take once the recovery target is + reached. The default is pause, which means recovery will + be paused. promote means the recovery process will finish + and the server will start to accept connections. + Finally shutdown will stop the server after reaching the + recovery target. + + + The intended use of the pause setting is to allow queries + to be executed against the database to check if this recovery target + is the most desirable point for recovery. + The paused state can be resumed by + using pg_wal_replay_resume() (see + ), which then + causes recovery to end. If this recovery target is not the + desired stopping point, then shut down the server, change the + recovery target settings to a later target and restart to + continue recovery. + + + The shutdown setting is useful to have the instance ready + at the exact replay point desired. The instance will still be able to + replay more WAL records (and in fact will have to replay WAL records + since the last checkpoint next time it is started). + + + Note that because recovery.signal will not be + removed when recovery_target_action is set to shutdown, + any subsequent start will end with immediate shutdown unless the + configuration is changed or the recovery.signal + file is removed manually. + + + This setting has no effect if no recovery target is set. + If is not enabled, a setting of + pause will act the same as shutdown. + + + + + + + @@ -3203,11 +3574,11 @@ Master Server application_name setting of the standby, as set in the standby's connection information. In case of a physical replication standby, this should be set in the primary_conninfo - setting in recovery.conf; the default - is walreceiver. For logical replication, this can - be set in the connection information of the subscription, and it - defaults to the subscription name. For other replication stream - consumers, consult their documentation. + setting; the default is walreceiver. + For logical replication, this can be set in the connection + information of the subscription, and it defaults to the + subscription name. For other replication stream consumers, + consult their documentation. This parameter specifies a list of standby servers using @@ -3350,6 +3721,79 @@ Standby Servers + + primary_conninfo (string) + + primary_conninfo configuration parameter + + + + + Specifies a connection string to be used for the standby server + to connect with a sending server. This string is in the format + described in . If any option is + unspecified in this string, then the corresponding environment + variable (see ) is checked. If the + environment variable is not set either, then + defaults are used. + + + The connection string should specify the host name (or address) + of the sending server, as well as the port number if it is not + the same as the standby server's default. + Also specify a user name corresponding to a suitably-privileged role + on the sending server (see + ). + A password needs to be provided too, if the sender demands password + authentication. It can be provided in the + primary_conninfo string, or in a separate + ~/.pgpass file on the standby server (use + replication as the database name). + Do not specify a database name in the + primary_conninfo string. + + + This parameter can only be set at server start. + This setting has no effect if the server is not in standby mode. + + + + + primary_slot_name (string) + + primary_slot_name configuration parameter + + + + + Optionally specifies an existing replication slot to be used when + connecting to the sending server via streaming replication to control + resource removal on the upstream node + (see ). + This parameter can only be set at server start. + This setting has no effect if primary_conninfo is not + set. + + + + + + promote_trigger_file (string) + + promote_trigger_file configuration parameter + + + + + Specifies a trigger file whose presence ends recovery in the + standby. Even if this value is not set, you can still promote + the standby using pg_ctl promote or calling + pg_promote. + This parameter can only be set at server start. + + + + hot_standby (boolean) @@ -3543,6 +3987,67 @@ Standby Servers + + recovery_min_apply_delay (integer) + + recovery_min_apply_delay configuration parameter + + + + + By default, a standby server restores WAL records from the + sending server as soon as possible. It may be useful to have a time-delayed + copy of the data, offering opportunities to correct data loss errors. + This parameter allows you to delay recovery by a fixed period of time, + measured in milliseconds if no unit is specified. For example, if + you set this parameter to 5min, the standby will + replay each transaction commit only when the system time on the standby + is at least five minutes past the commit time reported by the master. + + + It is possible that the replication delay between servers exceeds the + value of this parameter, in which case no delay is added. + Note that the delay is calculated between the WAL time stamp as written + on master and the current time on the standby. Delays in transfer + because of network lag or cascading replication configurations + may reduce the actual wait time significantly. If the system + clocks on master and standby are not synchronized, this may lead to + recovery applying records earlier than expected; but that is not a + major issue because useful settings of this parameter are much larger + than typical time deviations between servers. + + + The delay occurs only on WAL records for transaction commits. + Other records are replayed as quickly as possible, which + is not a problem because MVCC visibility rules ensure their effects + are not visible until the corresponding commit record is applied. + + + The delay occurs once the database in recovery has reached a consistent + state, until the standby is promoted or triggered. After that the standby + will end recovery without further waiting. + + + This parameter is intended for use with streaming replication deployments; + however, if the parameter is specified it will be honored in all cases. + + hot_standby_feedback will be delayed by use of this feature + which could lead to bloat on the master; use both together with care. + + + + Synchronous replication is affected by this setting when synchronous_commit + is set to remote_apply; every COMMIT + will need to wait to be applied. + + + + + This parameter can only be set at server start. + + + + diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml index 48ac14a838..0a10df6402 100644 --- a/doc/src/sgml/filelist.sgml +++ b/doc/src/sgml/filelist.sgml @@ -42,7 +42,6 @@ - diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index d730261742..09c77db045 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -19094,7 +19094,7 @@ Backup Control Functions pg_create_restore_point creates a named write-ahead log record that can be used as recovery target, and returns the corresponding write-ahead log location. The given name can then be used with - to specify the point up to which + to specify the point up to which recovery will proceed. Avoid creating multiple restore points with the same name, since recovery will stop at the first one whose name matches the recovery target. diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml index faf8e71854..d727d141f9 100644 --- a/doc/src/sgml/high-availability.sgml +++ b/doc/src/sgml/high-availability.sgml @@ -618,7 +618,7 @@ Standby Server Operation In standby mode, the server continuously applies WAL received from the master server. The standby server can read WAL from a WAL archive - (see ) or directly from the master + (see ) or directly from the master over a TCP connection (streaming replication). The standby server will also attempt to restore any WAL found in the standby cluster's pg_wal directory. That typically happens after a server @@ -645,7 +645,7 @@ Standby Server Operation Standby mode is exited and the server switches to normal operation when pg_ctl promote is run or a trigger file is found - (trigger_file). Before failover, + (promote_trigger_file). Before failover, any WAL immediately available in the archive or in pg_wal will be restored, but no attempt is made to connect to the master. @@ -686,10 +686,9 @@ Setting Up a Standby Server To set up the standby server, restore the base backup taken from primary - server (see ). Create a recovery - command file recovery.conf in the standby's cluster data - directory, and turn on standby_mode. Set - restore_command to a simple command to copy files from + server (see ). Create a file + standby.signal in the standby's cluster data + directory. Set to a simple command to copy files from the WAL archive. If you plan to have multiple standby servers for high availability purposes, set recovery_target_timeline to latest, to make the standby server follow the timeline change @@ -699,7 +698,7 @@ Setting Up a Standby Server Do not use pg_standby or similar tools with the built-in standby mode - described here. restore_command should return immediately + described here. should return immediately if the file does not exist; the server will retry the command again if necessary. See for using tools like pg_standby. @@ -708,11 +707,11 @@ Setting Up a Standby Server If you want to use streaming replication, fill in - primary_conninfo with a libpq connection string, including + with a libpq connection string, including the host name (or IP address) and any additional details needed to connect to the primary server. If the primary needs a password for authentication, the password needs to be specified in - primary_conninfo as well. + as well. @@ -724,7 +723,7 @@ Setting Up a Standby Server If you're using a WAL archive, its size can be minimized using the parameter to remove files that are no + linkend="guc-archive-cleanup-command"/> parameter to remove files that are no longer required by the standby server. The pg_archivecleanup utility is designed specifically to be used with archive_cleanup_command in typical single-standby @@ -735,9 +734,8 @@ Setting Up a Standby Server - A simple example of a recovery.conf is: + A simple example of configuration is: -standby_mode = 'on' primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass options=''-c wal_sender_timeout=5000''' restore_command = 'cp /path/to/archive/%f %p' archive_cleanup_command = 'pg_archivecleanup /path/to/archive %r' @@ -793,8 +791,8 @@ Streaming Replication To use streaming replication, set up a file-based log-shipping standby server as described in . The step that turns a file-based log-shipping standby into streaming replication - standby is setting primary_conninfo setting in the - recovery.conf file to point to the primary server. Set + standby is setting the primary_conninfo setting + to point to the primary server. Set and authentication options (see pg_hba.conf) on the primary so that the standby server can connect to the replication pseudo-database on the primary @@ -854,14 +852,14 @@ Authentication The host name and port number of the primary, connection user name, - and password are specified in the recovery.conf file. + and password are specified in the . The password can also be set in the ~/.pgpass file on the standby (specify replication in the database field). For example, if the primary is running on host IP 192.168.1.50, port 5432, the account name for replication is foo, and the password is foopass, the administrator - can add the following line to the recovery.conf file on the + can add the following line to the postgresql.conf file on the standby: @@ -973,10 +971,8 @@ Configuration Example (1 row) To configure the standby to use this slot, primary_slot_name - should be configured in the standby's recovery.conf. - Here is a simple example: + should be configured on the standby. Here is a simple example: -standby_mode = 'on' primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass' primary_slot_name = 'node_a_slot' @@ -1474,11 +1470,10 @@ Failover To trigger failover of a log-shipping standby server, run pg_ctl promote, call pg_promote, or create a trigger file with the file name and path specified by the - trigger_file setting in - recovery.conf. If you're planning to use + promote_trigger_file. If you're planning to use pg_ctl promote or to call pg_promote to fail over, - trigger_file is not required. If you're + promote_trigger_file is not required. If you're setting up the reporting servers that are only used to offload read-only queries from the primary, not for high availability purposes, you don't need to promote it. @@ -1491,11 +1486,8 @@ Alternative Method for Log Shipping An alternative to the built-in standby mode described in the previous sections is to use a restore_command that polls the archive location. - This was the only option available in versions 8.4 and below. In this - setup, set standby_mode off, because you are implementing - the polling required for standby operation yourself. See the - module for a reference - implementation of this. + This was the only option available in versions 8.4 and below. See the + module for a reference implementation of this. @@ -1522,14 +1514,13 @@ Alternative Method for Log Shipping The magic that makes the two loosely coupled servers work together is simply a restore_command used on the standby that, when asked for the next WAL file, waits for it to become available from - the primary. The restore_command is specified in the - recovery.conf file on the standby server. Normal recovery + the primary. Normal recovery processing would request a file from the WAL archive, reporting failure if the file was unavailable. For standby processing it is normal for the next WAL file to be unavailable, so the standby must wait for - it to appear. For files ending in + it to appear. For files ending in .history there is no need to wait, and a non-zero return - code must be returned. A waiting restore_command can be + code must be returned. A waiting restore_command can be written as a custom script that loops after polling for the existence of the next WAL file. There must also be some way to trigger failover, which should interrupt the restore_command, break the loop and @@ -1611,9 +1602,8 @@ Implementation Begin recovery on the standby server from the local WAL - archive, using a recovery.conf that specifies a - restore_command that waits as described - previously (see ). + archive, using restore_command that waits + as described previously (see ). @@ -2108,7 +2098,7 @@ Administrator's Overview If hot_standby is on in postgresql.conf - (the default value) and there is a recovery.conf + (the default value) and there is a standby.signal file present, the server will run in Hot Standby mode. However, it may take some time for Hot Standby connections to be allowed, because the server will not accept connections until it has completed diff --git a/doc/src/sgml/pgstandby.sgml b/doc/src/sgml/pgstandby.sgml index 2cc58fe356..d8aded4384 100644 --- a/doc/src/sgml/pgstandby.sgml +++ b/doc/src/sgml/pgstandby.sgml @@ -47,7 +47,7 @@ Description To configure a standby server to use pg_standby, put this into its - recovery.conf configuration file: + postgresql.conf configuration file: restore_command = 'pg_standby archiveDir %f %p %r' diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml index 0070603fc3..142799316a 100644 --- a/doc/src/sgml/postgres.sgml +++ b/doc/src/sgml/postgres.sgml @@ -158,7 +158,6 @@ Server Administration &maintenance; &backup; &high-availability; - &recovery-config; &monitoring; &diskusage; &wal; diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml deleted file mode 100644 index a2bdffda94..0000000000 --- a/doc/src/sgml/recovery-config.sgml +++ /dev/null @@ -1,510 +0,0 @@ - - - - Recovery Configuration - - - configuration - of recovery - of a standby server - - - - This chapter describes the settings available in the - recovery.confrecovery.conf - file. They apply only for the duration of the - recovery. They must be reset for any subsequent recovery you wish to - perform. They cannot be changed once recovery has begun. - - - - Settings in recovery.conf are specified in the format - name = 'value'. One parameter is specified per line. - Hash marks (#) designate the rest of the - line as a comment. To embed a single quote in a parameter - value, write two quotes (''). - - - - A sample file, share/recovery.conf.sample, - is provided in the installation's share/ directory. - - - - - Archive Recovery Settings - - - - restore_command (string) - - restore_command recovery parameter - - - - - The local shell command to execute to retrieve an archived segment of - the WAL file series. This parameter is required for archive recovery, - but optional for streaming replication. - Any %f in the string is - replaced by the name of the file to retrieve from the archive, - and any %p is replaced by the copy destination path name - on the server. - (The path name is relative to the current working directory, - i.e., the cluster's data directory.) - Any %r is replaced by the name of the file containing the - last valid restart point. That is the earliest file that must be kept - to allow a restore to be restartable, so this information can be used - to truncate the archive to just the minimum required to support - restarting from the current restore. %r is typically only - used by warm-standby configurations - (see ). - Write %% to embed an actual % character. - - - - It is important for the command to return a zero exit status - only if it succeeds. The command will be asked for file - names that are not present in the archive; it must return nonzero - when so asked. Examples: - -restore_command = 'cp /mnt/server/archivedir/%f "%p"' -restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows - - An exception is that if the command was terminated by a signal (other - than SIGTERM, which is used as part of a - database server shutdown) or an error by the shell (such as command - not found), then recovery will abort and the server will not start up. - - - - - - archive_cleanup_command (string) - - archive_cleanup_command recovery parameter - - - - - This optional parameter specifies a shell command that will be executed - at every restartpoint. The purpose of - archive_cleanup_command is to provide a mechanism for - cleaning up old archived WAL files that are no longer needed by the - standby server. - Any %r is replaced by the name of the file containing the - last valid restart point. - That is the earliest file that must be kept to allow a - restore to be restartable, and so all files earlier than %r - may be safely removed. - This information can be used to truncate the archive to just the - minimum required to support restart from the current restore. - The module - is often used in archive_cleanup_command for - single-standby configurations, for example: -archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r' - Note however that if multiple standby servers are restoring from the - same archive directory, you will need to ensure that you do not delete - WAL files until they are no longer needed by any of the servers. - archive_cleanup_command would typically be used in a - warm-standby configuration (see ). - Write %% to embed an actual % character in the - command. - - - If the command returns a nonzero exit status then a warning log - message will be written. An exception is that if the command was - terminated by a signal or an error by the shell (such as command not - found), a fatal error will be raised. - - - - - - recovery_end_command (string) - - recovery_end_command recovery parameter - - - - - This parameter specifies a shell command that will be executed once only - at the end of recovery. This parameter is optional. The purpose of the - recovery_end_command is to provide a mechanism for cleanup - following replication or recovery. - Any %r is replaced by the name of the file containing the - last valid restart point, like in . - - - If the command returns a nonzero exit status then a warning log - message will be written and the database will proceed to start up - anyway. An exception is that if the command was terminated by a - signal or an error by the shell (such as command not found), the - database will not proceed with startup. - - - - - - - - - - - Recovery Target Settings - - - By default, recovery will recover to the end of the WAL log. The - following parameters can be used to specify an earlier stopping point. - At most one of recovery_target, - recovery_target_lsn, recovery_target_name, - recovery_target_time, or recovery_target_xid - can be used; if more than one of these is specified in the configuration - file, the last entry will be used. - - - - - recovery_target = 'immediate' - - recovery_target recovery parameter - - - - - This parameter specifies that recovery should end as soon as a - consistent state is reached, i.e. as early as possible. When restoring - from an online backup, this means the point where taking the backup - ended. - - - Technically, this is a string parameter, but 'immediate' - is currently the only allowed value. - - - - - - recovery_target_name (string) - - recovery_target_name recovery parameter - - - - - This parameter specifies the named restore point (created with - pg_create_restore_point()) to which recovery will proceed. - - - - - - recovery_target_time (timestamp) - - recovery_target_time recovery parameter - - - - - This parameter specifies the time stamp up to which recovery - will proceed. - The precise stopping point is also influenced by - . - - - - - - recovery_target_xid (string) - - recovery_target_xid recovery parameter - - - - - This parameter specifies the transaction ID up to which recovery - will proceed. Keep in mind - that while transaction IDs are assigned sequentially at transaction - start, transactions can complete in a different numeric order. - The transactions that will be recovered are those that committed - before (and optionally including) the specified one. - The precise stopping point is also influenced by - . - - - - - - recovery_target_lsn (pg_lsn) - - recovery_target_lsn recovery parameter - - - - - This parameter specifies the LSN of the write-ahead log location up - to which recovery will proceed. The precise stopping point is also - influenced by . This - parameter is parsed using the system data type - pg_lsn. - - - - - - - The following options further specify the recovery target, and affect - what happens when the target is reached: - - - - - recovery_target_inclusive (boolean) - - recovery_target_inclusive recovery parameter - - - - - Specifies whether to stop just after the specified recovery target - (true), or just before the recovery target - (false). - Applies when , - , or - is specified. - This setting controls whether transactions - having exactly the target WAL location (LSN), commit time, or transaction ID, respectively, will - be included in the recovery. Default is true. - - - - - - recovery_target_timeline (string) - - recovery_target_timeline recovery parameter - - - - - Specifies recovering into a particular timeline. The default is - to recover along the same timeline that was current when the - base backup was taken. Setting this to latest recovers - to the latest timeline found in the archive, which is useful in - a standby server. Other than that you only need to set this parameter - in complex re-recovery situations, where you need to return to - a state that itself was reached after a point-in-time recovery. - See for discussion. - - - - - - recovery_target_action (enum) - - recovery_target_action recovery parameter - - - - - Specifies what action the server should take once the recovery target is - reached. The default is pause, which means recovery will - be paused. promote means the recovery process will finish - and the server will start to accept connections. - Finally shutdown will stop the server after reaching the - recovery target. - - - The intended use of the pause setting is to allow queries - to be executed against the database to check if this recovery target - is the most desirable point for recovery. - The paused state can be resumed by - using pg_wal_replay_resume() (see - ), which then - causes recovery to end. If this recovery target is not the - desired stopping point, then shut down the server, change the - recovery target settings to a later target and restart to - continue recovery. - - - The shutdown setting is useful to have the instance ready - at the exact replay point desired. The instance will still be able to - replay more WAL records (and in fact will have to replay WAL records - since the last checkpoint next time it is started). - - - Note that because recovery.conf will not be renamed when - recovery_target_action is set to shutdown, - any subsequent start will end with immediate shutdown unless the - configuration is changed or the recovery.conf file is - removed manually. - - - This setting has no effect if no recovery target is set. - If is not enabled, a setting of - pause will act the same as shutdown. - - - - - - - - - - Standby Server Settings - - - - standby_mode (boolean) - - standby_mode recovery parameter - - - - - Specifies whether to start the PostgreSQL server as - a standby. If this parameter is on, the server will - not stop recovery when the end of archived WAL is reached, but - will keep trying to continue recovery by fetching new WAL segments - using restore_command - and/or by connecting to the primary server as specified by the - primary_conninfo setting. - - - - - primary_conninfo (string) - - primary_conninfo recovery parameter - - - - - Specifies a connection string to be used for the standby server - to connect with the primary. This string is in the format - described in . If any option is - unspecified in this string, then the corresponding environment - variable (see ) is checked. If the - environment variable is not set either, then - defaults are used. - - - The connection string should specify the host name (or address) - of the primary server, as well as the port number if it is not - the same as the standby server's default. - Also specify a user name corresponding to a suitably-privileged role - on the primary (see - ). - A password needs to be provided too, if the primary demands password - authentication. It can be provided in the - primary_conninfo string, or in a separate - ~/.pgpass file on the standby server (use - replication as the database name). - Do not specify a database name in the - primary_conninfo string. - - - This setting has no effect if standby_mode is off. - - - - - primary_slot_name (string) - - primary_slot_name recovery parameter - - - - - Optionally specifies an existing replication slot to be used when - connecting to the primary via streaming replication to control - resource removal on the upstream node - (see ). - This setting has no effect if primary_conninfo is not - set. - - - - - trigger_file (string) - - trigger_file recovery parameter - - - - - Specifies a trigger file whose presence ends recovery in the - standby. Even if this value is not set, you can still promote - the standby using pg_ctl promote or calling - pg_promote. - This setting has no effect if standby_mode is off. - - - - - - recovery_min_apply_delay (integer) - - recovery_min_apply_delay recovery parameter - - - - - By default, a standby server restores WAL records from the - primary as soon as possible. It may be useful to have a time-delayed - copy of the data, offering opportunities to correct data loss errors. - This parameter allows you to delay recovery by a fixed period of time, - measured in milliseconds if no unit is specified. For example, if - you set this parameter to 5min, the standby will - replay each transaction commit only when the system time on the standby - is at least five minutes past the commit time reported by the master. - - - It is possible that the replication delay between servers exceeds the - value of this parameter, in which case no delay is added. - Note that the delay is calculated between the WAL time stamp as written - on master and the current time on the standby. Delays in transfer - because of network lag or cascading replication configurations - may reduce the actual wait time significantly. If the system - clocks on master and standby are not synchronized, this may lead to - recovery applying records earlier than expected; but that is not a - major issue because useful settings of this parameter are much larger - than typical time deviations between servers. - - - The delay occurs only on WAL records for transaction commits. - Other records are replayed as quickly as possible, which - is not a problem because MVCC visibility rules ensure their effects - are not visible until the corresponding commit record is applied. - - - The delay occurs once the database in recovery has reached a consistent - state, until the standby is promoted or triggered. After that the standby - will end recovery without further waiting. - - - This parameter is intended for use with streaming replication deployments; - however, if the parameter is specified it will be honored in all cases. - - hot_standby_feedback will be delayed by use of this feature - which could lead to bloat on the master; use both together with care. - - - - Synchronous replication is affected by this setting when synchronous_commit - is set to remote_apply; every COMMIT - will need to wait to be applied. - - - - - - - - - - diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml index c9f6ce4bb3..57dc83b620 100644 --- a/doc/src/sgml/ref/pg_basebackup.sgml +++ b/doc/src/sgml/ref/pg_basebackup.sgml @@ -214,10 +214,11 @@ Options - Write a minimal recovery.conf in the output + Create standby.signal and append connection settings + to postgresql.auto.conf in the output directory (or into the base archive file when using tar format) to ease setting up a standby server. - The recovery.conf file will record the connection + The postgresql.auto.conf file will record the connection settings and, if specified, the replication slot that pg_basebackup is using, so that the streaming replication will use the same settings later on. @@ -470,7 +471,7 @@ Options replication slot. If the base backup is intended to be used as a streaming replication standby using replication slots, it should then use the same replication slot name - in recovery.conf. That way, it is ensured that + in . That way, it is ensured that the server does not remove any necessary WAL data in the time between the end of the base backup and the start of streaming replication. diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml index e2662bbf81..53a64ee29e 100644 --- a/doc/src/sgml/ref/pg_rewind.sgml +++ b/doc/src/sgml/ref/pg_rewind.sgml @@ -69,7 +69,8 @@ Description target cluster ran for a long time after the divergence, the old WAL files might no longer be present. In that case, they can be manually copied from the WAL archive to the pg_wal directory, or - fetched on startup by configuring recovery.conf. The use of + fetched on startup by configuring or + . The use of pg_rewind is not limited to failover, e.g. a standby server can be promoted, run some write transactions, and then rewinded to become a standby again. @@ -83,8 +84,9 @@ Description pg_rewind was run, and therefore could not be copied by the pg_rewind session, it must be made available when the target server is started. This can be done by creating a - recovery.conf file in the target data directory with a - suitable restore_command. + recovery.signal file in the target data directory + and configuring suitable + in postgresql.conf. diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml index 4117a4392c..a3d3538b28 100644 --- a/doc/src/sgml/ref/pgarchivecleanup.sgml +++ b/doc/src/sgml/ref/pgarchivecleanup.sgml @@ -39,7 +39,7 @@ Description To configure a standby server to use pg_archivecleanup, put this into its - recovery.conf configuration file: + postgresql.conf configuration file: archive_cleanup_command = 'pg_archivecleanup archivelocation %r' @@ -47,7 +47,7 @@ Description files should be removed. - When used within , all WAL files + When used within , all WAL files logically preceding the value of the %r argument will be removed from archivelocation. This minimizes the number of files that need to be retained, while preserving crash-restart capability. Use of diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml index 2d722b2e79..978fa252e4 100644 --- a/doc/src/sgml/ref/pgupgrade.sgml +++ b/doc/src/sgml/ref/pgupgrade.sgml @@ -506,7 +506,7 @@ Save configuration files Save any configuration files from the old standbys' configuration directories you need to keep, e.g. postgresql.conf, - recovery.conf, because these will be overwritten or + pg_hba.conf, because these will be overwritten or removed in the next step. diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml index de7779f622..aacdd360e0 100644 --- a/doc/src/sgml/release-10.sgml +++ b/doc/src/sgml/release-10.sgml @@ -7303,7 +7303,7 @@ Replication and Recovery Allow specification of the recovery stopping point by Log Sequence Number (LSN) in - recovery.conf + recovery.conf (Michael Paquier) diff --git a/doc/src/sgml/release-9.1.sgml b/doc/src/sgml/release-9.1.sgml index e6ce80032f..9c4082b6c4 100644 --- a/doc/src/sgml/release-9.1.sgml +++ b/doc/src/sgml/release-9.1.sgml @@ -9811,7 +9811,7 @@ Recovery Control These named restore points can be specified as recovery targets using the new recovery.conf setting - recovery_target_name. + recovery_target_name. @@ -9843,8 +9843,7 @@ Recovery Control - Allow recovery.conf + Allow recovery.conf to use the same quoting behavior as postgresql.conf (Dimitri Fontaine) diff --git a/doc/src/sgml/release-9.4.sgml b/doc/src/sgml/release-9.4.sgml index 50442e98b4..51375cebe4 100644 --- a/doc/src/sgml/release-9.4.sgml +++ b/doc/src/sgml/release-9.4.sgml @@ -5350,7 +5350,7 @@ Changes - Ignore parameter until + Ignore parameter until recovery has reached a consistent state (Michael Paquier) @@ -10869,8 +10869,8 @@ Migration to Version 9.4 - Use the last specified recovery - target parameter if multiple target parameters are specified + Use the last specified recovery + target parameter if multiple target parameters are specified (Heikki Linnakangas) @@ -10889,7 +10889,7 @@ Migration to Version 9.4 User commands that did their own quote preservation might need adjustment. This is likely to be an issue for commands used in - , , + , , and COPY TO/FROM PROGRAM. @@ -11510,7 +11510,7 @@ Replication and Recovery - Add recovery parameter + Add recovery parameter to delay replication (Robert Haas, Fabrízio de Royes Mello, Simon Riggs) @@ -11523,7 +11523,7 @@ Replication and Recovery - Add + Add option to stop WAL recovery as soon as a consistent state is reached (MauMau, Heikki Linnakangas) @@ -11559,8 +11559,7 @@ Replication and Recovery - Report failure return codes from external recovery commands + Report failure return codes from external recovery commands (Peter Eisentraut) diff --git a/doc/src/sgml/release-9.5.sgml b/doc/src/sgml/release-9.5.sgml index ccd8eee3e3..da9f2f6979 100644 --- a/doc/src/sgml/release-9.5.sgml +++ b/doc/src/sgml/release-9.5.sgml @@ -7305,7 +7305,7 @@ Changes - Ignore parameter until + Ignore parameter until recovery has reached a consistent state (Michael Paquier) @@ -9096,9 +9096,9 @@ Replication and Recovery 2015-03-15 [51c11a7] Andres..: Remove pause_at_recovery_target recovery.conf s.. --> - Add recovery.conf + Add recovery.conf parameter recovery_target_action + linkend="guc-recovery-target-action">recovery_target_action to control post-recovery activity (Petr Jelínek) @@ -9200,7 +9200,7 @@ Replication and Recovery --> Allow recovery.conf's primary_conninfo setting to + linkend="guc-primary-conninfo">primary_conninfo setting to use connection URIs, e.g. postgres:// (Alexander Shulgin) diff --git a/doc/src/sgml/release.sgml b/doc/src/sgml/release.sgml index c4e763a043..4055adf88f 100644 --- a/doc/src/sgml/release.sgml +++ b/doc/src/sgml/release.sgml @@ -5,8 +5,7 @@ &<> use & escapes PostgreSQL -postgresql.conf, pg_hba.conf, - recovery.conf +postgresql.conf, pg_hba.conf [A-Z][A-Z_ ]+[A-Z_] , , , [A-Za-z_][A-Za-z0-9_]+() \-\-?[A-Za-z_]+[-A-Za-z_]*