Re: parallelizing the archiver

From: "Bossart, Nathan" <bossartn(at)amazon(dot)com>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Robert Haas <robertmhaas(at)gmail(dot)com>, Julien Rouhaud <rjuju123(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: parallelizing the archiver
Date: 2021-10-18 23:24:58
Message-ID: 8B7BF404-29D4-4662-A2DF-1AC4C98463EB@amazon.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 10/6/21, 1:34 PM, "Magnus Hagander" <magnus(at)hagander(dot)net> wrote:
> I definitely think that's the way to go. It gives a single path for everything which makes it simpler in the most critical parts. And once you have picked an implementation other than it, you're now completely rid of the old implementation. And of course the good old idea that having an extension already using the API is a good way to show that the API is in a good place.
>
> As much as I dislike our current interface in archive_command, and would like to see it go away completely, I do believe we need to ship something that has it - if nothing else then for backwards compatibility. But an extension like this would also make it easier to eventually, down the road, deprecate this solution.
>
> Oh, and please put said implementation in a better place than contrib :)

I've attached an attempt at moving the archive_command logic to its
own module and replacing it with a hook. This was actually pretty
straightforward.

I think the biggest question is where to put the archive_command
module, which I've called shell_archive. The only existing directory
that looked to me like it might work is src/test/modules. It might be
rather bold to relegate this functionality to a test module so
quickly, but on the other hand, perhaps it's the right thing to do
given we intend to deprecate it in the future. I'm curious what
others think about this.

I'm still working on the documentation updates, which are quite
extensive. I haven't included any of those in the patch yet.

Nathan

Attachment Content-Type Size
v2-0001-Replace-archive_command-with-a-hook.patch application/octet-stream 20.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bossart, Nathan 2021-10-18 23:30:36 Re: relation OID in ReorderBufferToastReplace error message
Previous Message Alvaro Herrera 2021-10-18 22:23:50 Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory