Re: RFC: PostgreSQL Storage I/O Transformation Hooks

On 28/12/2025 5:51 PM, Henson Choi wrote:
> Hi Konstantin,
>
> I understand the decorator pattern, and yes, it can work for some cases.
> But decorators can only intercept at the beginning and end of functions.
>
> Looking at the actual hook locations in md.c:
>
> - mdextend_pre_hook: after error checks, before file open → Decorator
> possible
> - mdwrite_pre_hook: after assertions, before I/O loop → Decorator possible
> - mdread_post_hook: inside the segment loop → Decorator NOT possible
>
> The mdreadv() function, introduced in PostgreSQL 17 as part of the
> vectored I/O API, processes multiple blocks in a loop that respects
> segment boundaries. The decryption hook must be called inside this loop,
> after each segment's FileReadV() completes. A decorator wrapping mdreadv()
> from the outside cannot access this internal loop timing.
>
> With the SMGR decorator approach, the extension developer must:
> - Track upstream md.c changes
> - Replicate the internal loop logic to find the right decryption point
>
> With hooks, the extension developer only needs to:
> - Implement encrypt() and decrypt()
>
> Regarding encryption+compression: that's a valid use case for SMGR,
> but our primary concern is different. In South Korea, government
> regulations require the use of nationally-approved cryptographic
> algorithms (such as ARIA, SEED). This means organizations often cannot
> adopt foreign TDE solutions, regardless of their technical merit.
>
> We need a simple, stable hook interface that allows local security
> experts to integrate these required algorithms - experts who understand
> cryptography but not PostgreSQL storage internals.
>
> If both approaches can coexist, why not provide hooks for the simple
> case and SMGR for the complex case?
>
> Best regards,
> Henson Choi

Hi Henson,

Thank you for explanations.
I personally do not like hooks, I considered them as some kind of
crutches which are needed to fix some problems with existed APIs:)
But them are quite popular in Postgres and really make it extensible.

The task of transparent data encryption is really very important for
Postgres (if for some reasons it can not be done at file system level).
If we need to add more hooks to make it possible to add to Postgres,
then dozen of yet another hooks may be acceptable...

I have not investigated it precisely, may be you are right that it is
possible to implement transparent encryption using using decorator
approach and custom SMGR. Frankly speaking I am quite upset how AIO was
added to PG18. It introduces orthogonal hierarchy to SMGR and cause
some  tight dependencies between this two modules which makes extension
of any of them problematic if ever possible (i.e. if I want to add my
storage manager and make AIO use it to access files system, rather than
calling pread/pwrite directly). I am not sure that AIO can not be added
through SMGR hierarchy (certainly by extending this interface), but it
is certainly separate store having no relation to the topic of this
discussion.

So I can assume that current coupling of AIO with SMGR makes it not
possible to plugin transparent encryption rather than adding this hooks.
Still not quite sure that proposed set of hooks is absolutely necessary
and sufficient...