Re: crash-safe visibility map, take three

On Wed, Dec 1, 2010 at 3:31 PM, Jeff Davis <pgsql(at)j-davis(dot)com> wrote:
> On Wed, 2010-12-01 at 11:25 -0500, Robert Haas wrote:
>> 1. Every time we observe a page as all-visible, (a) set the
>> PD_ALL_VISIBLE bit on the page, without bumping the LSN;
>
> ...
>
>> 2. Every time we observe a page as all-visible, (a) set the
>> PD_ALL_VISIBLE bit on the page, without bumping the LSN,
>
> My concern here is that both of these proposals introduce something new
> that is hint-bit-like, in the sense that we change a data page and
> potentially write it to disk without necessarily writing WAL. This will
> make it even harder to do CRCs in the future, which are also an
> important feature.

Well, there's an intermediate concept between "logged" and "unlogged",
which is an update that is WAL-logged but doesn't require torn-page
protection because writing some but not all of the 512-byte sectors in
the page won't break anything. We're using the LSN to track both
whether the page is logged and whether the update necessitates
torn-page protection. That might be fixable.

As for CRCs, there's a pretty direct chain of inference here:

1. CRCs are hard (really impossible) because we have hint bits.
2. Hint bits are necessary because an old XID can't be viewed as
guaranteed committed.
3. An old XID can't be viewed as guaranteed committed because we clean
up aborted transactions lazily rather than eagerly.

Changing (3) would amount to a rewrite of our whole MVCC architecture.
If we switched from per-tuple MVCC based on XIDs to per-page MVCC
based on LSNs and a rollback segment, all of this stuff would go out
the window. Hint bits, gone. Anti-wraparound VACUUM, gone. CRCs,
feasible. Visibility map... we might still need that, but the
page-level bits go away.

Of course, it would also create new problems.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company