Re: MaxOffsetNumber for Table AMs

On Tue, May 4, 2021 at 9:24 PM Peter Geoghegan <pg(at)bowt(dot)ie> wrote:
> Here is my concern: I have an obligation to make it clear that I think
> that you really ought to straighten out this business with
> generalizing TIDs before too long. Not because I say so, but because
> it's holding up progress in general. If you aren't getting cooperation
> from people who work on indexing (could be somebody else), then
> consider the possibility that this business with TIDs and bitmap scans
> has a lot to do with it. Most people are not as outspoken as I am.

It seems to me that we're doing a lot of disagreeing given that, as I
see it, there are only relatively minor differences between the
positions of the various people here. Andres and I are, I think,
relatively convinced that variable-width TIDs would let us do things
that aren't otherwise possible, and that those things are potentially
useful and I would even venture to say cool. I don't believe you
disagree with that, but you think it's going to be too much work to
implement. Fair enough; anyone can try it who is interested and see
how far they get. Anyone who thinks it's going to be impossibly hard
probably will prefer not to try, and that's OK too.

But if we take that off the table, what about a less-ambitious
generalization of the TID mechanism? I can't really see anyone putting
up a serious argument against allowing all 48 bits of space available
in the existing TID format to be used which, as Jeff points out, is
not currently the case. So anyone who wants to try to write that patch
is free to do so. I don't have a clear idea how to make that work, to
be honest, but my limited supply of ideas need not prevent anyone else
from trying theirs.

There might be some slight disagreement about whether it's useful to
generalize TIDs from a 48-bit address space to a 64-bit address space
without making it fully general. Like Andres, I am unconvinced that's
meaningfully easier, and I am convinced that it's meaningfully less
good, but other people can disagree and that's fine. I'm perfectly
willing to change my opinion if somebody shows up with a patch that
demonstrates the value of this approach.

The main point here is one that I think you made a few emails back:
the limitations of the current system are defined by what will
actually work with the code as it exists today, not some mailing list
discussion. It's too early for the project to commit to stability in
this area; we have not managed to get a single AM apart from heapam
into core, and that situation doesn't appear likely to change in the
near future. If and when we have say 5 of those we can probably
articulate some intelligent ideas about what we think the patterns
that need to hold for future AMs are, but it's reckless to extrapolate
from 1 working example, and right now that's all we have.

--
Robert Haas
EDB: http://www.enterprisedb.com