Re: parallelizing the archiver

Greetings,

* Bossart, Nathan (bossartn(at)amazon(dot)com) wrote:
> On 10/4/21, 7:21 PM, "Stephen Frost" <sfrost(at)snowman(dot)net> wrote:
> > This has something we've contemplated quite a bit and the last thing
> > that I'd want to have is a requirement to configure a whole bunch of
> > additional parameters to enable this. Why do we need to have some many
> > new GUCs? I would have thought we'd probably be able to get away with
> > just having the appropriate hooks and then telling folks to load the
> > extension in shared_preload_libraries..
>
> That would certainly simplify my patch quite a bit. I'll do it this
> way in the next revision.
>
> > As for the hooks themselves, I'd certainly hope that they'd be designed
> > to handle batches of WAL rather than individual ones as that's long been
> > one of the main issues with the existing archive command approach. I
> > appreciate that maybe that's less of an issue with a shared library but
> > it's still something to consider.
>
> Will do. This seems like it should be easier with the hook because we
> can provide a way to return which files were successfully archived.

It's also been discussed, at least around the water cooler (as it were
in pandemic times- aka our internal slack channels..) that the existing
archive command might be reimplemented as an extension using these. Not
sure if that's really necessary but it was a thought. In any case,
thanks for working on this!

Stephen