Re: [HACKERS] The dangers of "-F"

At 11:40 AM 6/23/99 -0400, Bruce Momjian wrote:

>Writing the buffers to a file, and making sure they are on the disk are
>different issues. Also, fsync only comes into play in an OS crash, so
>if that only happens once a year, and you are willing to restore from
>tape in that case (or check the integrity of the data on reboot), -F
>may be fine.

Ironically, I ran all day yesterday with -F and my nightly
dump failed on table "foo", "couldn't read block 0".

I've seen this once before without use of -F so I think it's
mere coincidence.

I realize that writing buffers to a file and making sure they're
on disk are two different issues. My point is that without the
fsynch, the backend loses control over the order in which blocks
are written to the disk.

For instance, if there are assumptions that all data blocks are
written before this fact is recorded in a log file, then
"write data blocks" "fsynch" "write log" "fsynch" doesn't break
that assumption, where "write data blocks" (no fsynch) "write log"
might, as the operating system's free to write the "write log"
blocks to disk before any of the data blocks are (though an
LRU algorithm most likely wouldn't). You could end up in a
case where the log records a successful write of data, without
any data actually being on disk.

I don't know how postgres works internally. So my question is
really "are any such assumptions broken by the use of -F, and
does breaking such assumptions lead to a more serious form
of failure if there's a crash?"

I agree that the risks of running -F are low with reliable
hardware and a UPS. I'm just trying to get a handle on just
what a user might be facing in terms of corruption compared
to a crash with fsynch'ing enabled. I can live with "the
database might well become corrupted and you'll have to
reload your latest dump".

My current plan is to implement a set of queries that do
fairly detailed consistency checks on my database every
night, before doing the nightly dump and copy to a second
machine, as well as each time I restart the web server
(typically only after crashes). In this way I'll know
quickly if any harm's been done after a crash, I'll have
some assurance the database is in good shape before dumps
(my code, not just the backend, might have bugs!), etc.

- Don Baccus, Portland OR <dhogaza(at)pacifier(dot)com>
Nature photos, on-line guides, and other goodies at
http://donb.photo.net