Re: 64-bit XIDs again

On 07/31/2015 12:29 AM, Josh Berkus wrote:
> On 07/30/2015 07:24 AM, Heikki Linnakangas wrote:
>>
>> I think we should move to 64-bit XIDs in in-memory structs snapshots,
>> proc array etc. And expand clog to handle 64-bit XIDs. But keep the
>> xmin/xmax fields on heap pages at 32-bits, and add an epoch-like field
>> to the page header so that logically the xmin/xmax fields on the page
>> are 64 bits wide, but physically stored in 32 bits. That's possible as
>> long as no two XIDs on the same page are more than 2^31 XIDs apart. So
>> you still need to freeze old tuples on the page when that's about to
>> happen, but it would make it possible to have more than 2^32 XID
>> transactions in the clog. You'd never be forced to do anti-wraparound
>> vacuums, you could just let the clog grow arbitrarily large
>
> When I introduced the same idea a few years back, having the clog get
> arbitrarily large was cited as a major issue. I was under the
> impression that clog size had some major performance impacts.

Well, sure, if you don't want the clog to grow arbitrarily large, then
you need to freeze. And most people would want to freeze regularly, to
keep the clog size in check. The point is that you wouldn't *have* to do
so at any particular time. You would never be up against the wall, in
the "you must freeze now or your database will shut down" situation.

I'm not sure what performance impact a very large clog might have. It
takes some disk space (1 GB per 4 billion XIDs), and caching it takes
some memory. And there is a small fixed number of CLOG buffers in shared
memory. But I don't think there's any particularly nasty problem there.

- Heikki