[PATCH v1 1/1] Add GENERATED ALWAYS AS ROW IDENTITY columns

Hi hackers,

This is something I have been thinking of for some time and here is a
proof of concept implementation to verify the advantages I have hoped
it will have.

What it is
========

A row identity type using a 64-bit expanded tuple ID as the identity,
can do direct access, with HOT chains following and backup index
access as backup.

It is also a quick PoC which hopefully can be simplified in the number
of ways including the amount of code changes.

What I hope to get out of this is
=======================

- immediate no-downtime ALTER TABLE ... ADD COLUMN .. GENERATED ALWAYS
AS ROW IDENTITY
- much faster point lookups by avoiding index access in most cases
- not needing the index maintenance in most cases, much cheaper in
remaining cases
- should be especially useful in many graph query cases

What is done in the PoC:
==================

Implement GENERATED ALWAYS AS ROW IDENTITY (and the rowidentity type
shorthand) for zero-byte on-disk in-place row identity columns with
hybrid direct-TID and partial B-Tree index access.

- Grammar & catalog: Support GENERATED ALWAYS AS ROW IDENTITY and
rowidentity column definitions, ATTRIBUTE_IDENTITY_ROW ('r') in
pg_attribute.attidentity, and partition_id_bits / partition_bits /
partition_id reloptions.
- 64-bit key layout & zero-byte home elision: Encode keys from
(partition_id, block_number, root_offset_number) when at home,
eliding the 8-byte int8 payload via t_bits = 0 (0 bytes stored on
disk, 0 index entries) and synthesizing the Datum during C and LLVM
JIT tuple deformation using HeapTupleData.t_rootoff.
- Off-page movement, repatriation & two-phase VACUUM: Materialize the
8-byte identity value inline and index it in the backing partial
B-Tree (WHERE id IS NOT NULL) only when a tuple moves off its home
page/partition, anchoring the home slot with LP_REDIRECT_OFFPAGE_*
line pointers, repatriating back to (B_0, O_0) when space permits,
and reclaiming orphaned anchors across a two-phase VACUUM pass.
- Hybrid TidScan (RowIdScan), directed INSERT/COPY, table rewrite &
amcheck: Support O(1) direct buffer lookups with partial B-Tree
fallback for =, IN, = ANY, cross-type int2/int4/int8 equality, and
prepared statements; directed home-slot placement on OVERRIDING
SYSTEM VALUE and COPY; home-elision preservation in VACUUM FULL and
CLUSTER; and heap/btree verification in contrib/amcheck.
---
contrib/amcheck/expected/check_heap.out | 36 +
contrib/amcheck/sql/check_heap.sql | 25 +
contrib/amcheck/verify_heapam.c | 23 +
src/backend/access/common/heaptuple.c | 80 ++
src/backend/access/common/reloptions.c | 58 +-
src/backend/access/common/tupdesc.c | 39 +-
src/backend/access/heap/heapam.c | 1051 ++++++++++++++++++--
src/backend/access/heap/heapam_handler.c | 175 +++-
src/backend/access/heap/heapam_indexscan.c | 5 +-
src/backend/access/heap/heapam_xlog.c | 32 +-
src/backend/access/heap/pruneheap.c | 99 +-
src/backend/access/heap/rewriteheap.c | 45 +
src/backend/access/heap/vacuumlazy.c | 328 +++++-
src/backend/catalog/index.c | 16 +-
src/backend/catalog/information_schema.sql | 4 +-
src/backend/commands/copyfrom.c | 5 +
src/backend/commands/copyfromparse.c | 14 +
src/backend/commands/indexcmds.c | 14 +-
src/backend/commands/tablecmds.c | 563 ++++++++++-
src/backend/executor/execIndexing.c | 33 +-
src/backend/executor/execMain.c | 13 +
src/backend/executor/execTuples.c | 102 +-
src/backend/executor/execUtils.c | 2 +
src/backend/executor/nodeTidscan.c | 488 ++++++++-
src/backend/jit/llvm/llvmjit_deform.c | 35 +-
src/backend/jit/llvm/meson.build | 1 +
src/backend/optimizer/path/costsize.c | 28 +-
src/backend/optimizer/path/tidpath.c | 150 ++-
src/backend/optimizer/util/plancat.c | 2 +-
src/backend/parser/gram.y | 16 +
src/backend/parser/parse_type.c | 4 +
src/backend/parser/parse_utilcmd.c | 194 +++-
src/backend/rewrite/rewriteHandler.c | 40 +-
src/backend/storage/page/bufpage.c | 3 +-
src/backend/utils/adt/ri_triggers.c | 3 +-
src/backend/utils/cache/lsyscache.c | 30 +
src/backend/utils/cache/relcache.c | 91 +-
src/backend/utils/sort/tuplesortvariants.c | 13 +-
src/bin/pg_dump/pg_dump.c | 22 +-
src/bin/psql/describe.c | 2 +
src/include/access/heapam.h | 68 ++
src/include/access/htup.h | 3 +-
src/include/access/htup_details.h | 157 +++
src/include/access/tupdesc.h | 5 +
src/include/catalog/pg_attribute.h | 1 +
src/include/executor/tuptable.h | 10 +-
src/include/storage/itemid.h | 19 +
src/include/utils/lsyscache.h | 1 +
src/include/utils/rel.h | 26 +
src/include/utils/relcache.h | 1 +
src/test/regress/expected/row_identity.out | 822 +++++++++++++++
src/test/regress/parallel_schedule | 3 +-
src/test/regress/sql/row_identity.sql | 485 +++++++++
53 files changed, 5282 insertions(+), 203 deletions(-)
create mode 100644 src/test/regress/expected/row_identity.out
create mode 100644 src/test/regress/sql/row_identity.sql

-----
Hannu