Re: 64 bit transaction id

On Tue, Nov 5, 2019 at 09:34:52AM +1300, Thomas Munro wrote:
> On Tue, Nov 5, 2019 at 8:45 AM Tomas Vondra
> > Agreed. I think code complexity is part of the trade-off. IMO it's fine
> > to hack existing heap AM initially, and only explore the separate AM if
> > that turns out to be promising.
>
> I thought a bit about how to make a minimally-diffferent-from-heap
> non-freezing table AM using 64 bit xids, as a thought experiment when
> trying to understand or explain to others what zheap is about.
> Committed transactions are easy (you don't have to freeze fxid
> references from the ancient past because they don't wrap around so
> they always look old), but how do you deal with *aborted* transactions
> when truncating the CLOG (given that our current rule is "if it's
> before the CLOG begins, it must be committed")? I see three
> possibilities: (1) don't truncate the CLOG anymore (use 64 bit
> addressing and let it leak disk forever, like we did before commit
> 2589735d and later work), (2) freeze aborted transactions only, using
> a wraparound vacuum (and now you have failed, if the goal was to avoid
> having to scan all tuples periodically to freeze stuff, though
> admittedly it will require less IO to freeze only the aborted
> transactions), (3) go and remove aborted fxid references eagerly, when
> you roll back (this could be done using the undo technology that we
> have been developing to support zheap). Another way to explain (3) is
> that this hypothetical table AM, let's call it "yheap", takes the
> minimum parts of the zheap technology stack required to get rid of
> vacuum-for-wraparound, without doing in-place updates or any of that
> hard stuff. To make this really work you'd also have to deal with
> multixacts, which also require freezing. If that all sounds too
> complicated, you're back to (2) which seems a bit weak to me. Or
> perhaps I'm missing something?

The above is a very good summary of the constraints that have led to our
current handling of XID wraparound. If we are concerned about excessive
vacuum freeze overhead, why is the default autovacuum_freeze_max_age =
200000000 so low? That causes feezing when the pg_xact directory holds
200 million xids or 50 megabytes of xid status?

As far as I understand it, we cause the database to stop writes when the
xid counter gets within 2 billion xids of the current transaction
counter, so 200 million is only 1/10th to that limit, and even then, I
am not sure why we couldn't make it stop writes at 3 billion or
something. My point is that increasing the default
autovacuum_freeze_max_age value seems like an easy way to reduce vacuum
freeze. (While, the visibility map helps avoid vacuum freeze from
reading all heap pages, and we still need to read all index pages.)

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +