On 28/12/2025 9:49 AM, Henson Choi wrote:
>
>
> RFC: PostgreSQL Storage I/O Transformation Hooks
>
>
> Infrastructure for a Technical Protocol Between RDBMS Core and
> Data Security Experts
>
> *Author:* Henson Choi assam258(at)gmail(dot)com
>
> *Date:* 2025-12-28
>
> *PostgreSQL Version:* master (Development)
>
> ------------------------------------------------------------------------
>
>
> 1. Summary & Motivation
>
> This RFC proposes the introduction of minimal hooks into the
> PostgreSQL storage layer and the addition of a *Transformation ID*
> field to the |PageHeader|.
>
>
> A Diplomatic Protocol Between Expert Groups
>
> The core motivation of this proposal is *“Separation of Concerns and
> Mutual Respect.”*
>
> Historically, discussions around Transparent Data Encryption (TDE)
> have often felt like putting security experts on trial in a foreign
> court—specifically, the “Court of RDBMS.” It is time to treat them not
> as defendants to be judged by database-specific rules, but as an
> *equal neighboring community* with their own specialized sovereignty.
>
> *The issue has never been a failure of technology, but rather a
> misplacement of the focal point.* While previous discussions were
> mired in the technicalities of “how to hardcode encryption into the
> core,” this proposal shifts the debate toward an architectural
> solution: “what interface the core should provide to external experts.”
>
> * *RDBMS Experts* provide a trusted pipeline responsible for data
> I/O paths and consistency.
> * *Security Experts* take responsibility for the specialized domain
> of encryption algorithms and key management.
>
> This hook system functions as a *Technical Protocol*—a high-level
> agreement that allows these two expert groups to exchange data
> securely without encroaching on each other’s territory.
>
> ------------------------------------------------------------------------
>
>
> 2. Design Principles
>
> 1. *Delegation of Authority:* The core remains independent of
> specific encryption standards, providing a “free territory” where
> security experts can respond to an ever-changing security landscape.
> 2. *Diplomatic Convention:* The Transformation ID acts as a
> communication protocol between the engine and the extension. The
> engine uses this ID to identify the state of the data and hands
> over control to the appropriate expert (the extension).
> 3. *Minimal Interference:* Overhead is kept near zero when hooks are
> not in use, ensuring the native performance of the PostgreSQL engine.
>
> ------------------------------------------------------------------------
>
>
> 3. Proposal Specifications
>
>
> 3.1 The Interface (Hook Points)
>
> We allow intervention by security experts through five contact points
> along the I/O path:
>
> * *Read/Write Hooks:* |mdread_post|, |mdwrite_pre|, |mdextend_pre|
> (Transformation of the data area)
> * *WAL Hooks:* |xlog_insert_pre|, |xlog_decode_pre| (Transformation
> of transaction logs)
>
>
> 3.2 The Protocol Identifier (PageHeader Transformation ID)
>
> We allocate 5 bits of |pd_flags| to define the “Security State” of a
> page. This serves as a *Status Message* sent by the security expert to
> the engine, utilized for key versioning and as a migration marker.
>
> ------------------------------------------------------------------------
>
>
> 4. Reference Implementation: |contrib/test_tde|
>
>
> A Standard Code of Conduct for Security Experts
>
> This reference implementation exists not as a commercial product, but
> to define the *Standards of the Diplomatic Protocol* that
> encryption/decryption experts must follow when entering the PostgreSQL
> domain.
>
> 1. *Deterministic IV Derivation:* Demonstrates how to achieve
> cryptographic safety by trusting unique values provided by the
> engine (e.g., LSN).
> 2. *Critical Section Safety:* Defines memory management regulations
> that security logic must follow within “Critical Sections” to
> maintain system stability.
> 3. *Hook Chaining:* Demonstrates a cooperative structure that allows
> peaceful coexistence with other expert tools (e.g., compression,
> auditing).
>
> ------------------------------------------------------------------------
>
>
> 5. Scope
>
> * *In-Scope:* Backend hook infrastructure, Transformation ID field,
> and reference code demonstrating diplomatic protocol compliance.
> * *Out-of-Scope:* Specific Key Management Systems (KMS), selection
> of specific cryptographic algorithms, and integration with
> external tools.
>
> This proposal represents a strategic diplomatic choice: rather than
> the PostgreSQL core assuming all security responsibilities, it grants
> security experts a *sovereign territory through extensions* where they
> can perform at their best.
>
I wonder if instead of support a lot of extra hooks it will be better to
provide extensible SMGR API:
https://www.postgresql.org/message-id/flat/CAPP%3DHha_wV1MV9yR70QZ5pk5dtNP%2BbOyBiFxPmrMKqnQeKMAwQ%40mail.gmail.com#ab0da3412525c7501ea17f3d4c602bbf
It seems to be much more straightforward, convenient and flexible
mechanism than adding hooks, which can be used for many other purposes
except transparent encryption.