[RFC PATCH v3 0/7] Umbra: a remap-aware smgr prototype

Hi hackers,

This message continues the Umbra v3 P1-P7 review unit in the original
thread.

I mistakenly sent the August 8 v3 update as a separate thread. This
message continues the earlier Umbra submission; the material and
attachments are otherwise unchanged.

Since the prior submission, I have refactored Umbra into a
PostgreSQL-oriented three-slot physical-placement model and optimized
the implementation around this model. With full_page_writes=on, Umbra's
performance is now close to md with full_page_writes=off.

I have completed a separate audit of P1-P7, so I am submitting that
review unit now. Later parts of the implementation already exist, but
their audit is still in progress and they are deliberately excluded
from this review request.

The attached umbra-p1-p7-20260808.tar.gz contains P1-P7 only. Each
patch has its own commit message describing its capability, ownership
boundary, invariant, dependency, and non-goals.

SHA256 (umbra-p1-p7-20260808.tar.gz):

cc1ca7345b3d2503626643a825f57247414d6e12a5e84a15330c907c0b52931f

The review scope is:

1. P1-P7: smgr selection, the md-compatible physical umfile layer,
private MAP metadata bootstrap and residency, compatibility limits,
and the first working mapped layout.

Later Umbra work includes:

2. Selector-shift WAL, redo, and checkpoint publication and correctness.

3. Performance and operational optimization.

The first stage introduces the infrastructure and layout state required
by the later selector-shift work: smgr selection, umfile, private MAP
metadata, the initial mapped layout, and selector read/cache support.
Selector-shift WAL, checkpoint-overlap handling, compaction/restacking,
and other later mechanisms are not included in this review unit.

Umbra places logical-to-physical block layout below smgr. MAP is
Umbra-owned, relation-bound physical-layout metadata. Table and index
AMs continue to operate only on logical block numbers and do not
interpret MAP pages, physical slots, or selectors.

The model uses three physical slots for each logical page. The slots
retain the older page versions required for redo and rotate in order.
This increases physical page space, but allows Umbra to avoid the WAL
amplification caused by traditional full-page writes while
full_page_writes remains enabled.

Recent TPC-C results are below. The workload uses 800 warehouses and
runs for 20 minutes, with checkpoint_timeout=4min and
checkpoint_completion_target=0.9.

```
terminals md FPW=on md FPW=off Umbra FPW=on
-------------------------------------------------------
200 385616 614035 604807
400 279927 540942 568584
800 192126 420854 426757
```

WAL volume per New-Order is:

```
terminals md FPW=on md FPW=off Umbra FPW=on
-------------------------------------------------------
200 26.05 KiB 7.53 KiB 7.54 KiB
400 36.25 KiB 7.83 KiB 7.81 KiB
800 37.71 KiB 8.40 KiB 8.24 KiB
```

The complete benchmark summary and hardware report are attached.
md with full_page_writes=off is included only as an upper-bound and
sensitivity reference; it is not a correctness-equivalent
configuration.

The relatively stable branch for testing or inspecting the complete
design remains:

https://github.com/nayishan/postgre_umbra/tree/chunksplit-restack

Please reply in this thread so that the discussion remains in one
place.

Regards,
Mingwei Jia