Re: Hot Backup with rsync fails at pg_clog if under load

>
> The way forwards here is for someone to show the clog file that causes
> the error and find out why the call to read() fails.

Well let me hook ya up :) Here's the log, controldata, and erroring clog
attached

2011-10-26 08:44:15.419 MDT [1544]: [1-1] LOG: database system was
interrupted; last known up at 2011-10-26 08:43:54 MDT
2011-10-26 08:44:15.419 MDT [1544]: [2-1] LOG: creating missing WAL
directory "pg_xlog/archive_status"
2011-10-26 08:44:15.421 MDT [1544]: [3-1] LOG: entering standby mode
DEBUG: received replication command: IDENTIFY_SYSTEM
DEBUG: received replication command: START_REPLICATION 6/CE000000
2011-10-26 08:44:15.425 MDT [1545]: [1-1] LOG: streaming replication
successfully connected to primary
2011-10-26 08:44:16.246 MDT [1547]: [1-1] FATAL: the database system is
starting up
2011-10-26 08:44:17.501 MDT [1544]: [4-1] DEBUG: checkpoint record is at
6/D0BC36A0
2011-10-26 08:44:17.501 MDT [1544]: [5-1] DEBUG: redo record is at
6/CE0010E0; shutdown FALSE
2011-10-26 08:44:17.501 MDT [1544]: [6-1] DEBUG: next transaction ID:
0/10846655; next OID: 2826628
2011-10-26 08:44:17.501 MDT [1544]: [7-1] DEBUG: next MultiXactId: 1741;
next MultiXactOffset: 3498
2011-10-26 08:44:17.501 MDT [1544]: [8-1] DEBUG: oldest unfrozen
transaction ID: 1669, in database 1
2011-10-26 08:44:17.501 MDT [1544]: [9-1] DEBUG: transaction ID wrap limit
is 2147485316, limited by database with OID 1
2011-10-26 08:44:17.742 MDT [1544]: [10-1] DEBUG: resetting unlogged
relations: cleanup 1 init 0
2011-10-26 08:44:17.751 MDT [1544]: [11-1] DEBUG: initializing for hot
standby
2011-10-26 08:44:17.751 MDT [1544]: [12-1] FATAL: could not access status
of transaction 10846655
2011-10-26 08:44:17.751 MDT [1544]: [13-1] DETAIL: Could not read from file
"pg_clog/000A" at offset 90112: Success.
2011-10-26 08:44:17.751 MDT [1537]: [2-1] LOG: startup process (PID 1544)
exited with exit code 1
2011-10-26 08:44:17.751 MDT [1537]: [3-1] LOG: aborting startup due to
startup process failure
2011-10-26 08:44:17.843 MDT [1543]: [1-1] DEBUG: logger shutting down

pg_control version number: 903
Catalog version number: 201105231
Database system identifier: 5667259861501982685
Database cluster state: in production
pg_control last modified: Wed 26 Oct 2011 08:43:54 AM MDT
Latest checkpoint location: 6/D0BC36A0
Prior checkpoint location: 6/877B9950
Latest checkpoint's REDO location: 6/CE0010E0
Latest checkpoint's TimeLineID: 1
Latest checkpoint's NextXID: 0/10846655
Latest checkpoint's NextOID: 2826628
Latest checkpoint's NextMultiXactId: 1741
Latest checkpoint's NextMultiOffset: 3498
Latest checkpoint's oldestXID: 1669
Latest checkpoint's oldestXID's DB: 1
Latest checkpoint's oldestActiveXID: 10845042
Time of latest checkpoint: Wed 26 Oct 2011 08:43:44 AM MDT
Minimum recovery ending location: 0/0
Backup start location: 0/0
Current wal_level setting: hot_standby
Current max_connections setting: 600
Current max_prepared_xacts setting: 0
Current max_locks_per_xact setting: 256
Maximum data alignment: 8
Database block size: 8192
Blocks per segment of large relation: 131072
WAL block size: 8192
Bytes per WAL segment: 16777216
Maximum length of identifiers: 64
Maximum columns in an index: 32
Maximum size of a TOAST chunk: 1996
Date/time type storage: 64-bit integers
Float4 argument passing: by value
Float8 argument passing: by value