Re: 64 bit TID?

On Mon, Sep 13, 2021 at 5:36 PM Peter Geoghegan <pg(at)bowt(dot)ie> wrote:
> If somebody wants to make TIDs (or some generalized TID-like thing
> that tableam knows about) into logical identifiers, then they must
> also answer the question: identifiers of what?
>
> TIDs from Postgres heapam identify a physical version, or perhaps a
> HOT chain -- which is not how TIDs work in other DB systems that use a
> heap structure. This is the only reason why we can mostly think of
> indexes as data structures that don't need to be involved in
> concurrency control. Postgres index access methods don't usually need
> to know anything about locks that protect the logical structure of the
> database.

The 1993 paper "Options in Physical Database Design" gives a useful
overview of the challenges here. Especially for an extensibile system
like Postgres relative to a system with a traditional design
implementing classic ARIES.

I think that you need an ACM membership to get a copy. The relevant
section starts out like this:

"""
Item Representation
-------------------

Physical representation types for abstract data types
is only slowly gaining research attention for object-
oriented database systems but will likely become a
very important tuning option. Examples include sets
represented as bit maps, arrays, or lists and matrices
represented densely or sparsely, by row or by column
or as tiles, e.g. [MaV93]. The goal is to bring physical
data independence to object-oriented and scientific
databases and their applications.

Physical pointers, references, or object identifiers to
represent relationships support "navigation" through a
database, which is very good for single-instance
retrievals and often improves set matching, but also
creates a new type of updates, structural updates,
which may increase the complexity of concurrency
control and recovery [CSL90, ChK84, RoR85,
ShC90].
"""

This seems to be a fundamental trade-off that is tied inextricably to
the design of many other things.

That doesn't stop anybody from creating a column store using the
tableam. But it does mean that they will need to be very careful about
defining what exact "logical vs physical vs physiological" tradeoff
they've chosen. It's rather subtle stuff.

--
Peter Geoghegan