Re: AW: beta testing version

From: Bruce Guenter <bruceg(at)em(dot)ca>
To: "'pgsql-hackers(at)postgresql(dot)org'" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: AW: beta testing version
Date: 2000-12-06 17:49:10
Message-ID: 20001206114910.A19337@em.ca
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Wed, Dec 06, 2000 at 11:15:26AM -0500, Tom Lane wrote:
> Zeugswetter Andreas SB <ZeugswetterA(at)Wien(dot)Spardat(dot)at> writes:
> > Yes, but there would need to be a way to verify the last page or
> > record from txlog when running on crap hardware.
> How exactly *do* we determine where the end of the valid log data is,
> anyway?

I don't know how pgsql does it, but the only safe way I know of is to
include an "end" marker after each record. When writing to the log,
append the records after the last end marker, ending with another end
marker, and fdatasync the log. Then overwrite the previous end marker
to indicate it's not the end of the log any more and fdatasync again.

To ensure that it is written atomically, the end marker must not cross a
hardware sector boundary (typically 512 bytes). This can be trivially
guaranteed by making the marker a single byte.

Any other way I've seen discussed (here and elsewhere) either
- Requires atomic multi-sector writes, which are possible only if all
the sectors are sequential on disk, the kernel issues one large write
for all of them, and you don't powerfail in the middle of the write.
- Assume that a CRC is a guarantee. A CRC would be a good addition to
help ensure the data wasn't broken by flakey drive firmware, but
doesn't guarantee consistency.

--
Bruce Guenter <bruceg(at)em(dot)ca> http://em.ca/~bruceg/

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2000-12-06 17:54:06 Re: How To Log User Name + Database?
Previous Message Jeff Meeks 2000-12-06 17:44:40 Design and development info

Browse pgsql-hackers by date

  From Date Subject
Next Message Oleg Bartunov 2000-12-06 18:11:08 Re: CVS: miscadmin.h is missing
Previous Message Daniele Orlandi 2000-12-06 17:39:46 Re: AW: beta testing version