Re: ERROR: missing chunk number 0 for toast value 382548694 in pg_toast_847386

Hi Vibhor,

>> Not sure about above wrapper function. However, if you can share some
>> information from pg_log when you have started the restore with
>> backup_label information.

Here it is at the beginning:
[2011-02-25 09:40:11 EST] LOG: database system was interrupted; last known up at 2011-02-01 01:04:12 EST
[2011-02-25 09:40:11 EST] LOG: starting archive recovery
[2011-02-25 09:40:11 EST] LOG: restore_command = 'gunzip -c /mnt/dbsbackup/pg_xlog/%f.gz > %p'
gunzip: /mnt/dbsbackup/pg_xlog/00000001.history.gz: No such file or directory
[2011-02-25 09:40:12 EST] LOG: restored log file "0000000100006F720000006F.00024AE0.backup" from archive
[2011-02-25 09:40:12 EST] LOG: restored log file "0000000100006F7200000094" from archive
[2011-02-25 09:40:12 EST] LOG: restored log file "0000000100006F720000006F" from archive
[2011-02-25 09:40:12 EST] LOG: automatic recovery in progress
[2011-02-25 09:40:12 EST] LOG: redo starts at 6F72/6F024AE0, consistency will be reached at 70B1/B75C9AF0
[2011-02-25 09:40:14 EST] LOG: restored log file "0000000100006F7200000070" from archive
[2011-02-25 09:40:14 EST] LOG: restored log file "0000000100006F7200000071" from archive
[2011-02-25 09:40:15 EST] LOG: restored log file "0000000100006F7200000072" from archive

...etc...

We did reach consistency eventually:

[2011-02-25 21:29:28 EST] LOG: restored log file "00000001000070B1000000B5" from archive
[2011-02-25 21:29:29 EST] LOG: restored log file "00000001000070B1000000B6" from archive
[2011-02-25 21:29:29 EST] LOG: restored log file "00000001000070B1000000B7" from archive
[2011-02-25 21:29:29 EST] LOG: consistent recovery state reached
[2011-02-25 21:29:29 EST] LOG: restored log file "00000001000070B1000000B8" from archive
[2011-02-25 21:29:30 EST] LOG: restored log file "00000001000070B1000000B9" from archive
...

>> Try following steps:
>> 1. Untar all the gzipped WAL File in One Location
>> 2. Use Following restore command:
>> cp <WAL Location>/%f %p

Unfortunately it is not practical to do this, since there are many terabytes of WAL files. They are available on an NFS mounted volume which is accessible from the target machine.

What we can do (and will do if no better suggestions come forward) is to restore again using only the PITR files needed to achieve consistency, ie. up to 70B1/B75C9AF0, a much smaller quantity of WAL files.

>> We were concerned that shutting down / starting up while recovery is
>> ongoing might cause some problems, but the pg documentation indicates
>> this should be OK and we saw no cause for concern in the pg logs.
> What options have you used for shutting down?

We are using Centos and used the supplied init scripts - the actual command is: service postgresql stop, which translates to:

pg_ctl stop -D '$PGDATA' -s -m fast

Regards
Mike Harris