------------- Introduction. ------------- Operation log is designed to store information about critical system events (like pg_upgrade, pg_resetwal, pg_resetwal, etc.). This information is not interesting to the ordinary user, but it is very important for the vendor's technical support. An example: client complains about DB damage to technical support (damage was caused by pg_resetwal which was "silent" executed by one of administrators). Concepts. --------- * operation log is placed in the separate file 'global/pg_control_log' (log size is 8kB); * user can not modify the operation log; log can be changed by function call only (from postgres or from postgres utilities); * operation log is a ring buffer (with CRC-32 protection), deleting entries from the operation log is possible only when the buffer is overflowed; * SQL-function is used to read data of operation log. Example of operation log data. ------------------------------ >select * from pg_operation_log(); event |edition|version| lsn | last |count ------------+-------+-------+---------+----------------------+------ startup |vanilla|10.22.0|0/8000028|2022-11-18 23:06:27+03| 1 pg_upgrade |vanilla|15.0.0 |0/8000028|2022-11-18 23:06:27+03| 1 startup |vanilla|15.0.0 |0/80001F8|2022-11-18 23:11:53+03| 3 pg_resetwal|vanilla|15.0.0 |0/80001F8|2022-11-18 23:09:53+03| 2 (4 rows) Some details about inserting data to operation log. --------------------------------------------------- There are two modes of inserting information about events in the operation log. * MERGE mode (events "startup", "pg_resetwal", "pg_rewind"). We searches in ring buffer of operation log an event with the same type ("startup" for example) with the same version number. If event was found, we will increment event counter by 1 and update the date/time of event ("last" field) with the current value. If event was not found, we will add this event to the ring buffer (see INSERT mode). * INSERT mode (events "bootstrap", "pg_upgrade", "promoted"). We will add an event to the ring buffer (without searching). ------------------------- Operation log structures. ------------------------- The operation log is placed in the pg_control_log file (file size PG_OPERATION_LOG_FULL_SIZE=8192). Operation log is a ring buffer of 341 elements (24 bytes each) with header (8 bytes). а) Operation log element: typedef struct OperationLogData { uint8 ol_type; /* operation type */ uint8 ol_edition; /* postgres edition */ uint16 ol_count; /* number of operations */ uint32 ol_version; /* postgres version */ pg_time_t ol_timestamp; /* = int64, operation date/time */ XLogRecPtr ol_lsn; /* = uint64, last check point record ptr */ } OperationLogData; Description of fields: * ol_type - event type. The types are contained in an enum: typedef enum { OLT_BOOTSTRAP = 1, /* bootstrap */ OLT_STARTUP, /* server startup */ OLT_RESETWAL, /* pg_resetwal */ OLT_REWIND, /* pg_rewind */ OLT_UPGRADE, /* pg_upgrade */ OLT_PROMOTED, /* promoted */ OLT_NumberOfTypes /* should be last */ } ol_type_enum; * ol_edition - PostgreSQL edition (this field is important for custom PostgreSQL editions). The editions are contained in an enum: typedef enum { ED_PG_ORIGINAL = 0 /* Here can be custom editions */ } PgNumEdition; * ol_count - the number of fired events in case events of this type can be merged (otherwise 1). Maximum is 65536; * ol_version - version formed with using PG_VERSION_NUM (PG_VERSION_NUM*100; for example: 13000800 for v13.8). Two last digits can be used for patch version; * ol_timestamp - date/time of the last fired event in case events of this type can be merged (otherwise - the date/time of the event); * ol_lsn - "Latest checkpoint location" value (ControlFile->checkPoint) which contains in pg_control file at the time of the event. б) Operation log header: typedef struct OperationLogHeader { pg_crc32c ol_crc; /* CRC of operation log ... MUST BE FIRST! */ uint16 ol_first; /* position of first ring buffer element */ uint16 ol_count; /* number of elements in ring buffer */ } OperationLogHeader; Description of fields: * ol_crc - CRC of operation log (from position sizeof(pg_crc32c) to PG_OPERATION_LOG_FULL_SIZE=8192); * ol_first - 0-based position of the first element in the ring buffer; * ol_count - number of elements in the ring buffer. ----------------------- Event processing modes. ----------------------- There are two event processing modes for the operation log: typedef enum { OLM_MERGE = 1, /* insert element only if not exists element * with the same ol_type and ol_version; * otherwise update existing element */ OLM_INSERT, /* insert element into ring buffer 'as is' */ OLM_NumberOfModes /* should be last */ } ol_mode_enum; Description of processing modes: * OLM_MERGE - for a new event need to search in the ring buffer for an event of the same type with the same version number. If an event is found in the buffer, then it is modified: а) the ol_count field value is increased by 1 (until it not reached PG_UINT16_MAX=65535); б) the ol_timestamp field is modified with the current date/time. If event is not found then it added to the buffer (the same as OLM_INSERT mode). * OLM_INSERT - add an event to the ring buffer without searching first. Structure with modes for events: OperationLogTypeDesc OperationLogTypesDescs[] = { {OLT_BOOTSTRAP, OLM_INSERT, "bootstrap"}, {OLT_STARTUP, OLM_MERGE, "startup"}, {OLT_RESETWAL, OLM_MERGE, "pg_resetwal"}, {OLT_REWIND, OLM_MERGE, "pg_rewind"}, {OLT_UPGRADE, OLM_INSERT, "pg_upgrade"}, {OLT_PROMOTED, OLM_INSERT, "promoted"} }; ------------------------ Operation log functions. ------------------------ (см. файл src\include\common\controldata_utils.h) a) Getting operation log by DataDir: OperationLogBuffer * get_operation_log(const char *DataDir, bool *crc_ok_p); b) Writing operation log (log_buffer) to pg_control_log file by DataDir: void update_operation_log(const char *DataDir, OperationLogBuffer * log_buffer); c) Writing of event with type ol_type to operation log: void put_operation_log_element(const char *DataDir, ol_type_enum ol_type); d) Writing of event with type ol_type to operation log with edition (edition) and version number (version_num): void put_operation_log_element_version(const char *DataDir, ol_type_enum ol_type, PgNumEdition edition, uint32 version_num); e) Function returns the number of events in the operation log (log_buffer): uint16 get_operation_log_count(OperationLogBuffer *log_buffer); f) Function returns the description of the event with number num from the operation log (log_buffer): OperationLogData *get_operation_log_element(OperationLogBuffer *log_buffer, uint16 num); g) Function returns the name of event by type (ol_type). For example, for ol_type=OLT_BOOTSTRAP function returns "bootstrap". const char *get_operation_log_type_name(ol_type_enum ol_type); ------------------------------------------------- Reading the operation log using the SQL function. ------------------------------------------------- For new databases, the system function pg_operation_log() is automatically created. For existing databases, the pg_operation_log() function should be created manually: CREATE OR REPLACE FUNCTION pg_operation_log( OUT event text, OUT edition text, OUT version text, OUT lsn pg_lsn, OUT last timestamptz, OUT count int4) RETURNS SETOF record LANGUAGE INTERNAL STABLE STRICT PARALLEL SAFE AS 'pg_operation_log'; Example of using pg_operation_log() function: >select * from pg_operation_log(); event | edition | version | lsn | last | count ------------+---------+---------+-----------+------------------------+------- startup | vanilla | 10.22.0 | 0/8000028 | 2022-10-27 23:06:27+03 | 1 pg_upgrade | vanilla | 15.0.0 | 0/8000028 | 2022-10-27 23:06:27+03 | 1 startup | vanilla | 15.0.0 | 0/80001F8 | 2022-10-27 23:06:53+03 | 2 (3 rows) ---------------------- Feature of pg_upgrade. ---------------------- If the operation log is empty, then before writing event "pg_upgrade", the startup event with the previous version is written to the operation log. ---------------------- Feature of pg_rewind. ---------------------- pg_rewind copies all the contents of the catalog "global/" (include "global/pg_control_log") from source database to target database. New pg_rewind feature reads operation log of target database at the start of the pg_rewind work, adds "pg_rewind" event and stores the log in the target database at the end of the pg_rewind work.