Re: Refactoring the checkpointer's fsync request queue

Hi,

On 2019-02-22 10:18:57 -0800, Shawn Debnath wrote:
> I think using callbacks is the better path forward than having md or
> other components issue an invalidate request for each and every segment
> which can get quite heavy handed for large databases.

I'm not sure I buy this. Unlinking files isn't cheap, involves many disk
writes, etc. The cost of an inval request in comparison isn't
particularly large. Especially for relation-level (rather than database
level) truncation, per-segment invals will likely commonly be faster
than the sequential scan.

> At the time of smgrinit(), mdinit() would call into sync and register
> it's callbacks with an ID. We can use the same value that we are using
> for smgr_which to map the callbacks. Each fsync request will then also
> accompany this ID which the sync mechanism will use to call handlers for
> resolving forget requests or obtaining paths for files.

I'm not seeing a need to do this dynamically at runtime. Given that smgr
isn't extensible, why don't we just map callbacks (or even just some
switch based logic) based on some enum? Doing things at *init time has
more potential to go wrong, because say a preload_shared_library does
different things in postmaster than normal backends (in EXEC_BACKEND
cases).

Besides those two points, I think this is going in a good direction!

Greetings,

Andres Freund