Re: Adding new flags to XLogRecord

On Thu, 2008-09-18 at 08:38 -0400, Tom Lane wrote:
> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> > On Thu, 2008-09-18 at 12:40 +0100, Greg Stark wrote:
> >> Why bit just add a new bitfield for flags if we need them? I'm usually
> >> the one worried about data density so perhaps I should be on the other
> >> side of the fence here but I'm not sure. The conventional wisdom is
> >> that wal bandwidth is not a major issue.
>
> > In some cases, but my wish is also to minimise WAL volume as much as
> > possible.
>
> I'm with Greg on this one: baroque bit-squeezing schemes are a bad idea.

I'm easy, just trying to save bytes.

> You still haven't answered the question of what you need four more bits
> for (and why four more is all that anyone will ever need --- unless you
> can prove that, we might as well just add another flag field).

Just trying to avoid having same info on multiple threads.

I would like to use 2 flag bits for Hot Standby:

* flag1 to indicate the first WAL record for an xid. This allows Hot
Standby to efficiently manage a recovery snapshot.
* flag2 to indicate the first WAL record for a subxid. This will
indicate that there is another 4 byte xid at end of struc and before
backup blocks that holds parentxid for this subxid. That allows us to
maintain subtrans during Hot Standby.

If we do this the suggested way it will add X bytes for flag bits (2?)
to every WAL record, plus 4 bytes to every initial WAL record in a
subtransaction.

There will also be occasional WAL records to avoid race conditions, as
described in the thread on transaction snapshots and hot standby, though
those are rare. There are also some other WAL records that may need to
be written in other less common paths also, which will be individually
justified when I come to them.

The two bits I need will be set zero in xl_prev until we have written
4x10^18 bytes of WAL, or 4000 Petabytes. So we're not really ever like
to witness them set in our lifetimes for diagnostic purposes.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support