Re: Extensible storage manager API - smgr hooks

Anastasia Lubennikova писал 2021-06-30 00:49:
> Hi, hackers!
>
> Many recently discussed features can make use of an extensible storage
> manager API. Namely, storage level compression and encryption [1],
> [2], [3], disk quota feature [4], SLRU storage changes [5], and any
> other features that may want to substitute PostgreSQL storage layer
> with their implementation (i.e. lazy_restore [6]).
>
> Attached is a proposal to change smgr API to make it extensible. The
> idea is to add a hook for plugins to get control in smgr and define
> custom storage managers. The patch replaces smgrsw[] array and smgr_sw
> selector with smgr() function that loads f_smgr implementation.
>
> As before it has only one implementation - smgr_md, which is wrapped
> into smgr_standard().
>
> To create custom implementation, a developer needs to implement smgr
> API functions
> static const struct f_smgr smgr_custom =
> {
> .smgr_init = custominit,
> ...
> }
>
> create a hook function
>
> const f_smgr * smgr_custom(BackendId backend, RelFileNode rnode)
> {
> //Here we can also add some logic and chose which smgr to use
> based on rnode and backend
> return &smgr_custom;
> }
>
> and finally set the hook:
> smgr_hook = smgr_custom;
>
> [1]
> https://www.postgresql.org/message-id/flat/11996861554042351(at)iva4-dd95b404a60b(dot)qloud-c(dot)yandex(dot)net
> [2]
> https://www.postgresql.org/message-id/flat/272dd2d9.e52a.17235f2c050.Coremail.chjischj%40163.com
> [3] https://postgrespro.com/docs/enterprise/9.6/cfs
> [4]
> https://www.postgresql.org/message-id/flat/CAB0yre%3DRP_ho6Bq4cV23ELKxRcfhV2Yqrb1zHp0RfUPEWCnBRw%40mail.gmail.com
> [5]
> https://www.postgresql.org/message-id/flat/20180814213500.GA74618%4060f81dc409fc.ant.amazon.com
> [6]
> https://wiki.postgresql.org/wiki/PGCon_2021_Fun_With_WAL#Lazy_Restore
>
> --
>
> Best regards,
> Lubennikova Anastasia

Good day, Anastasia.

I also think smgr should be extended with different implementations
aside of md.
But which way concrete implementation will be chosen for particular
relation?
I believe it should be (immutable!) property of tablespace, and should
be passed
to smgropen. Patch in current state doesn't show clear way to distinct
different
implementations per relation.

I don't think patch should be that invasive. smgrsw could pointer to
array instead of static array as it is of now, and then reln->smgr_which
will remain with same meaning. Yep it then will need a way to select
specific
implementation, but something like `char smgr_name[NAMEDATALEN]` field
with
linear search in (i believe) small smgrsw array should be enough.

Maybe I'm missing something?

regards,
Sokolov Yura.