Re: Recover from corrupted database due to failing disk

From: Gionatan Danti <g(dot)danti(at)assyoma(dot)it>
To: Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com>, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Recover from corrupted database due to failing disk
Date: 2016-11-04 10:20:21
Message-ID: 7c7373bf-e7fb-d2d6-5631-98c5959891c6@assyoma.it
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

On 03/11/2016 14:20, Adrian Klaver wrote:
>
> The above does not make sense. You are having to recover because there
> was no backup and now you want to go forward without doing a backup?
>

Hi Adrian, no, I don't want go forward without backups ;)
Actually, the *first* thing I did after the vacuum completed was a full
cluster backup (via pg_dumpall), and I scheduled nightly backups as well.

Problem is this customer does not have another server were backups can
be restored and the entire production database migrated. In short, the
two possibilities I have are:

1) execute the vacuum (done), schedule regular dumps (done) and, if
something goes wrong, recover from backups;

2) execute the vacuum (done), do a manual backup (done), reinit
(remove/recreate) the entire cluster (not done) and restore from backups
(not done).

I strongly prefer to execute n.2 on another machine, so that production
is not impacted while the recovered backup can be througly tested.
If/when the backups are validated, I want to migrate all clients to the
new server (with RAID1 in place), and dismiss the old one.

Unfortuntaly I am working with incredible constrains from customer side;
even buying two SAS disks seems a problem. Moreover, as an external
consultant, I have basically no decision/buying power :|
What I can do (and I did) is to raise a very big red flag and let others
decide what to do.

The good thing is that zero_damaged_pages and vacuum did their works, as
now the database can be dumped and vacuumed with no (apparent) problems.

Thanks.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g(dot)danti(at)assyoma(dot)it - info(at)assyoma(dot)it
GPG public key ID: FF5F32A8

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Kim Rose Carlsen 2016-11-04 13:08:33 Re: How to hint 2 coulms IS NOT DISTINCT FROM each other
Previous Message Venkata B Nagothi 2016-11-04 09:21:48 Re: replication setup: advice needed