Re: recovery modules

On Fri, Jan 27, 2023 at 04:09:39PM -0800, Andres Freund wrote:
> I think it would be hard to write a good module that isn't just implementing
> the existing commands without it. Needing to clean up archives and reacting to
> the end of recovery seems a pretty core task.

FWIW, recovery_end_command is straight-forward as it is done by the
startup process, so that's an easy take. You could split the cake
into two parts, as well, aka first focus on restore_command and
recovery_end_command as a first step, then we could try to figure out
how archive_cleanup_command would fit in this picture with the
checkpointer or a different process. There are a bunch of deployments
where WAL archive retention is controlled by the age of the backups,
not by the backend deciding when these should be removed as a
checkpoint runs depending on the computed redo LSN, so recovery
modules would still be useful with just coverage for restore_command
and recovery_end_command.

> I was briefly wondering whether it'd be worth trying to offload things like
> archive_cleanup_command from checkpointer to a different process, for
> robustness. But given that it's pretty much required for performance that the
> module runs in the startup process, that ship probably has sailed.

Yeah, agreed that this could be interesting. That could leverage the
work of the checkpointer. Nathan has proposed a patch for that
recently, as far as I recall, to offload some tasks from the startup
and checkpointer processes:
https://commitfest.postgresql.org/41/3448/

So that pretty much goes into the same area?
--
Michael