Re: Extensible storage manager API - SMGR hook Redux

On Mon, 4 Dec 2023 at 22:21, Matthias van de Meent <
boekewurm+postgres(at)gmail(dot)com> wrote:

> On Mon, 4 Dec 2023 at 17:51, Kirill Reshke <reshkekirill(at)gmail(dot)com> wrote:
> >
> > So, 0002 patch uses the `get_tablespace` function, which searches
> Catalog to tablespace SMGR id. I wonder how `smgr_redo` would work with it?
>
> That's a very good point I hadn't considered in detail yet. Quite
> clearly, the current code is wrong in assuming that the catalog is
> accessible, and it should probably be stored in a way similar to
> pg_filenode.map in a file managed outside the buffer pool.
>
> Hmm, pg_filenode.map is a nice idea. So, simply maintain TableSpaceOId ->
smgr id mapping in a separate file and update the whole file on any
changes, right?
Looks reasonable to me, but it is clear that this solution can be really
slow in some patterns, like if we create many-many tablespaces(the way you
suggested it in the per-relation SMGR feature). Maybe we can store data in
files somehow separately, and only update one chunk per operation.

Anyway, if we use a `pg_filenode.map` - like solution, we need to reuse its
code infrasture, right? For example, it seems that code that calculates
checksums can be reused.
So, we need to refactor code here, define something like FileMap API maybe.
Or is it not really worth it? We can just write similar code twice.