Re: Allow WAL information to recover corrupted pg_controldata

From: Amit kapila <amit(dot)kapila(at)huawei(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Cédric Villemain <cedric(at)2ndquadrant(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, "'Robert Haas'" <robertmhaas(at)gmail(dot)com>
Subject: Re: Allow WAL information to recover corrupted pg_controldata
Date: 2012-06-17 05:03:27
Message-ID: 6C0B27F7206C9E4CA54AE035729E9C38285094A4@szxeml509-mbs
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> "Torn disk sector"? Please, this is nonsense. Disks cannot write half a sector and then stop.
What I was intended to say is corruption due to hardware or some other problem, not because when
Postgres is updating pg_control file. For example http://cquirke.mvps.org/9x/baddata.htm.

> Well, we invented pg_resetxlog with the thought that it might be useful
> for such situations, but I'm not sure offhand that we've ever seen a
> field report of corrupted pg_control files.
I have found few cases where people have tried to use pg_resetxlog due to hardware problems or missing pg_control file.
http://archives.postgresql.org/pgsql-performance/2004-06/msg00236.php
http://archives.postgresql.org/pgsql-general/2004-06/msg00173.php
http://archives.postgresql.org/pgsql-admin/2006-12/msg00205.php

> Case in point here is that it's not immediately obvious that we should trust
> the contents of WAL more than pg_control.
Agreed.

> At the moment I don't see that we have either (a) or (b), so I think
> it's pretty dubious to be making any changes of this sort.
As the chances of usecase for this feature are very less, So I will stop working on this feature.

________________________________________
From: Tom Lane [tgl(at)sss(dot)pgh(dot)pa(dot)us]
Sent: Saturday, June 16, 2012 12:11 PM
To: Amit kapila
Cc: Cédric Villemain; pgsql-hackers(at)postgresql(dot)org; 'Robert Haas'
Subject: Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata

Amit kapila <amit(dot)kapila(at)huawei(dot)com> writes:
>> AFAIR pg_controldata fit on a disk sector so it can not be half written.

> It can be corrupt due to some other reasons as well like torn disk sector.

"Torn disk sector"? Please, this is nonsense. Disks cannot write half
a sector and then stop. A sufficiently badly designed drive might
attempt to start a write when it didn't have enough power left to finish
... but the result of that would be a corrupt sector with a non-matching
CRC, not one that read back okay but contained erroneous data.

> The suggested patch improves the logic to recover corrupt control file. So that is the reason I felt it will be relevant to do this patch.

Well, we invented pg_resetxlog with the thought that it might be useful
for such situations, but I'm not sure offhand that we've ever seen a
field report of corrupted pg_control files. For instance, a quick
search in the archives for "incorrect checksum in control file" turns up
only cases of pilot error, such as supposing that a 32-bit database
could be used with a 64-bit server or vice versa. Actual hardware
failures on the pg_control file could be expected to result in something
like "could not read from control file: I/O error", which I find no
evidence for at all in the archives.

Before adding new code to improve the situation, it would be good to
have (a) evidence that there's a problem worth solving, and (b) a theory
as to what likely-to-occur cases the new code is going to make better,
while not making things worse in other likely-to-occur cases. Case in
point here is that it's not immediately obvious that we should trust
the contents of WAL more than pg_control --- the former gets a whole
lot more write traffic and hence has many more opportunities for
failure.

At the moment I don't see that we have either (a) or (b), so I think
it's pretty dubious to be making any changes of this sort.

regards, tom lane

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2012-06-17 06:54:35 Re: [PATCH] Support for foreign keys with arrays
Previous Message nik9000 2012-06-17 04:00:20 Re: Pg default's verbosity?