WAL recovery failure

From: Rick Dicaire <kritek(at)gmail(dot)com>
To: pgsql-admin(at)postgresql(dot)org
Subject: WAL recovery failure
Date: 2012-09-25 18:22:48
Message-ID: CA+Cc--f4LCRSM6fTrUodWBByYzVZNtCFT_7AYb2g9RMnhVyeGQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Hi folks, we're snapshotting at san level, two volumes containing a
postgres 9.1.5 database, and pg_xlog respectively.
This database cluster is a streaming replicant.
We're stopping the database completely, taking the snapshots,
restarting the database.
I export these snapshots as luns to a VM, mount the disks in the VM,
cp -a the database and pg_xlog contents to
a staging area. Start postgres using -D /staging/9.1/main. Postgres
fails to start.

>From log:

2012-09-25 12:41:30.147 CDT,,,9597,,5061ecca.257d,1,,2012-09-25
12:41:30 CDT,,0,LOG,00000,"database system was interrupted while in
recovery at 2012-09-25 12:18:06 CDT",,"This probably means that some
data is corrupted and you will have to use the last backup for
recovery.",,,,,,,""
2012-09-25 12:41:30.155 CDT,,,9597,,5061ecca.257d,2,,2012-09-25
12:41:30 CDT,,0,LOG,00000,"starting archive recovery",,,,,,,,,""
2012-09-25 12:41:30.354 CDT,,,9597,,5061ecca.257d,3,,2012-09-25
12:41:30 CDT,,0,LOG,00000,"restored log file
""000000010000074D000000EE"" from archive",,,,,,,,,""
2012-09-25 12:41:30.529 CDT,,,9597,,5061ecca.257d,4,,2012-09-25
12:41:30 CDT,,0,LOG,00000,"restored log file
""000000010000074D000000E2"" from archive",,,,,,,,,""
2012-09-25 12:41:30.529 CDT,,,9597,,5061ecca.257d,5,,2012-09-25
12:41:30 CDT,,0,PANIC,XX000,"unexpected pageaddr 74D/7C000000 in log
file 1869, segment 226, offset 0",,,,,,,,,""
2012-09-25 12:41:30.530 CDT,,,9595,,5061ecc8.257b,1,,2012-09-25
12:41:28 CDT,,0,LOG,00000,"startup process (PID 9597) was terminated
by signal 6: Aborted",,,,,,,,,""
2012-09-25 12:41:30.530 CDT,,,9595,,5061ecc8.257b,2,,2012-09-25
12:41:28 CDT,,0,LOG,00000,"terminating any other active server
processes",,,,,,,,,""

/staging/9.1/main/recovery.conf:

restore_command = 'cp /staging/9.1/main/pg_xlog/%f "%p"'

Does the PANIC shown in above log entry indicate a corrupt WAL file?

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Steve Crawford 2012-09-25 21:22:46 Upgrading and streaming replication
Previous Message P R 2012-09-25 17:19:52 WAL Log Processing For OLAP?