Re: parallelizing the archiver

On 10/19/21, 9:14 AM, "Bossart, Nathan" <bossartn(at)amazon(dot)com> wrote:
> My original proof-of-concept also attempted to handle a bunch of other
> shell command GUCs, but perhaps I'd better keep this focused on
> archive_command for now. What we do here could serve as an example of
> how to adjust the other shell command GUCs later on. I'll go ahead
> and rework my patch to look more like what is being discussed here,
> although I expect the exact design for the interface will continue to
> evolve based on the feedback in this thread.

Alright, I reworked the patch a bit to maintain backward
compatibility. My initial intent for 0001 was to just do a clean
refactor to move the shell archiving stuff to its own file. However,
after I did that, I realized that adding the hook wouldn't be too much
more work, so I did that as well. This seems to be enough to support
custom archiving modules. I included a basic example of such a module
in 0002. 0002 is included primarily for demonstration purposes.

I do wonder if there are some further enhancements we should make to
the archiving module interface. With 0001 applied, archive_command is
silently ignored if you've preloaded a library that uses the hook.
There's no way to indicate that you actually want to use
archive_command or that you want to use a specific library as the
archive library. On the other hand, just adding the hook keeps things
simple, and it doesn't preclude future improvements in this area.

Nathan