Re: archive modules

On 2021/11/02 3:54, Bossart, Nathan wrote:
> This thread is a continuation of the thread with the subject
> "parallelizing the archiver" [0]. That thread had morphed into an
> effort to allow creating archive modules, so I've created a new one to
> ensure that this topic has the proper visibility.

What is the main motivation of this patch? I was thinking that
it's for parallelizing WAL archiving. But as far as I read
the patch very briefly, WAL file name is still passed to
the archive callback function one by one.

Are you planning to extend this mechanism to other WAL
archiving-related commands like restore_command? I can imagine
that those who use archive library (rather than shell) would
like to use the same mechanism for WAL restore.

> I've attached the latest patch from the previous thread. This patch
> does a few things. First, it adds the archive_library GUC that
> specifies a library to use in place of archive_command. If
> archive_library is set to "shell" (the default), archive_command is
> still used. The archive_library is preloaded, so its _PG_init() can
> do anything that libraries loaded via shared_preload_libraries can do.
> Like logical decoding output plugins, archive modules must define an
> initialization function and some callbacks. The patch also introduces
> the basic_archive module to ensure test coverage on the new
> infrastructure.

I think that it's worth adding this module into core
rather than handling it as test module. It provides very basic
WAL archiving feature, but (I guess) it's enough for some users.

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION