BUG #5574: PANIC on hot-standby: heap_update_redo

The following bug has been logged online:

Bug reference: 5574
Logged by: Valentine Gogichashvili
Email address: valgog(at)gmail(dot)com
PostgreSQL version: 9.0beta3
Operating system: RedHat Linux (2.4.21-53.ELsmp)
Description: PANIC on hot-standby: heap_update_redo
Details:

A hot-standby and a source database are identical machines.

A hot-standby machine was recieving WAL files for 2 days now (omnipitr
scripts), and then it crashed with the following error:

2010-07-27 15:20:38.318 CEST,,,22151,,4c49acba.5687,5691,,2010-07-23
16:52:42 CEST,1/0,0,LOG,00000,"restored log file
""0000000100000024000000CA"" from archive",,,,,,,,,""
2010-07-27 15:21:38.430 CEST,,,22151,,4c49acba.5687,5692,,2010-07-23
16:52:42 CEST,1/0,0,LOG,00000,"restored log file
""0000000100000024000000CB"" from archive",,,,,,,,,""
2010-07-27 15:22:37.682 CEST,,,22151,,4c49acba.5687,5693,,2010-07-23
16:52:42 CEST,1/0,0,LOG,00000,"restored log file
""0000000100000024000000CC"" from archive",,,,,,,,,""
2010-07-27 15:23:37.224 CEST,,,22151,,4c49acba.5687,5694,,2010-07-23
16:52:42 CEST,1/0,0,LOG,00000,"restored log file
""0000000100000024000000CD"" from archive",,,,,,,,,""
2010-07-27 15:24:37.606 CEST,,,22151,,4c49acba.5687,5695,,2010-07-23
16:52:42 CEST,1/0,0,LOG,00000,"restored log file
""0000000100000024000000CE"" from archive",,,,,,,,,""
2010-07-27 15:25:38.118 CEST,,,22151,,4c49acba.5687,5696,,2010-07-23
16:52:42 CEST,1/0,0,LOG,00000,"restored log file
""0000000100000024000000CF"" from archive",,,,,,,,,""
2010-07-27 15:26:09.759 CEST,,,22151,,4c49acba.5687,5697,,2010-07-23
16:52:42 CEST,1/0,0,PANIC,XX000,"heap_update_redo: invalid max offset
number",,,,,"xlog redo update: rel 16384/31487/35687; tid 207118/46; new
207118/73",,,,""
2010-07-27 15:26:09.999 CEST,,,22148,,4c49acba.5684,2,,2010-07-23 16:52:42
CEST,,0,LOG,00000,"startup process (PID 22151) was terminated by signal 6:
Aborted",,,,,,,,,""
2010-07-27 15:26:09.999 CEST,,,22148,,4c49acba.5684,3,,2010-07-23 16:52:42
CEST,,0,LOG,00000,"terminating any other active server
processes",,,,,,,,,""

After retrying to startup the hot-standby again:

2010-07-27 16:00:38.300 CEST,,,2899,,4c4ee686.b53,1,,2010-07-27 16:00:38
CEST,,0,LOG,00000,"database system was interrupted while in recovery at log
time 2010-07-27 15:16:41 CEST",,"If this has occurred more than once some
data might be corrupted and you might need to choose an earlier recovery
target.",,,,,,,""
2010-07-27 16:00:38.300 CEST,,,2899,,4c4ee686.b53,2,,2010-07-27 16:00:38
CEST,,0,LOG,00000,"entering standby mode",,,,,,,,,""
2010-07-27 16:00:38.520 CEST,,,2899,,4c4ee686.b53,3,,2010-07-27 16:00:38
CEST,,0,LOG,00000,"restored log file ""0000000100000024000000C7"" from
archive",,,,,,,,,""
2010-07-27 16:00:39.050 CEST,,,2899,,4c4ee686.b53,4,,2010-07-27 16:00:38
CEST,1/0,0,LOG,00000,"redo starts at 24/C70046E8",,,,,,,,,""
2010-07-27 16:00:39.300 CEST,,,2899,,4c4ee686.b53,5,,2010-07-27 16:00:38
CEST,1/0,0,LOG,00000,"restored log file ""0000000100000024000000C8"" from
archive",,,,,,,,,""
2010-07-27 16:00:39.560 CEST,,,2899,,4c4ee686.b53,6,,2010-07-27 16:00:38
CEST,1/0,0,LOG,00000,"restored log file ""0000000100000024000000C9"" from
archive",,,,,,,,,""
2010-07-27 16:00:39.840 CEST,,,2899,,4c4ee686.b53,7,,2010-07-27 16:00:38
CEST,1/0,0,LOG,00000,"restored log file ""0000000100000024000000CA"" from
archive",,,,,,,,,""
2010-07-27 16:00:40.310 CEST,,,2899,,4c4ee686.b53,8,,2010-07-27 16:00:38
CEST,1/0,0,LOG,00000,"restored log file ""0000000100000024000000CB"" from
archive",,,,,,,,,""
2010-07-27 16:00:40.370 CEST,,,2899,,4c4ee686.b53,9,,2010-07-27 16:00:38
CEST,1/0,0,PANIC,XX000,"heap_update_redo: invalid lp",,,,,"xlog redo update:
rel 16384/31487/35687; tid 182931/32; new 185403/25",,,,""
2010-07-27 16:00:40.410 CEST,,,2896,,4c4ee686.b50,1,,2010-07-27 16:00:38
CEST,,0,LOG,00000,"startup process (PID 2899) was terminated by signal 6:
Aborted",,,,,,,,,""
2010-07-27 16:00:40.410 CEST,,,2896,,4c4ee686.b50,2,,2010-07-27 16:00:38
CEST,,0,LOG,00000,"terminating any other active server
processes",,,,,,,,,""

With best regards,

-- Valentine