Re: Adding new flags to XLogRecord

On Thu, 2008-09-18 at 13:56 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > I'd like to add some new flag bits to XLogRecord. (xlog.h)
> >
> > Where? xl_prev.
>
> I'm more curious about "What?" and "Why?", actually ;-),

Just trying to solve the egg/chicken problem of "I want to add a flag";
"but there are no flags available". I'm sure there's a few uses coming
up in synch replication also.

I want two flags for Hot Standby, but lets justify them on another post.

> > So I would like to propose that we ignore the top 4 bits in
> > xl_prev.xlogid when comparing values, rather than using all 32 bits for
> > comparison. That then frees up 4 new flag bits on XLogRecords. Changing
> > xl_prev handling is only required in 3 places, all in xlog.c, plus some
> > log outputs.
>
> Or, we could store only the delta between current record and the
> previous one. Assuming we know what the current record is, that wouldn't
> lose any precision. That way xl_prev only needs to be as big as the
> biggest possible WAL record we can have.
>
> Not that I think the precision in your scheme isn't enough, but I find
> the delta easier to comprehend.

That can't work, I know, that was my first thought.

The files are reused, so xl_prev protects against reusing a file and
then having a perfectly valid WAL record *after* the correct end of WAL
that makes it look like WAL continues. So a delta could be valid data
even though the record was invalid.

We don't want to zero the files cause that costs too much, so we have to
allow for seemingly correct data as well as correct data in WAL.

I think the xl_prev field could be removed completely when streaming,
except the new flags of course.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support