Re: parallelizing the archiver

From: "Bossart, Nathan" <bossartn(at)amazon(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Magnus Hagander <magnus(at)hagander(dot)net>, Stephen Frost <sfrost(at)snowman(dot)net>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Julien Rouhaud <rjuju123(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: parallelizing the archiver
Date: 2021-10-25 19:45:21
Message-ID: 0C272F63-37B1-43FC-AB5E-04A68ED4335C@amazon.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 10/25/21, 10:50 AM, "Bossart, Nathan" <bossartn(at)amazon(dot)com> wrote:
> On 10/25/21, 10:18 AM, "Robert Haas" <robertmhaas(at)gmail(dot)com> wrote:
>> On Mon, Oct 25, 2021 at 1:14 PM Bossart, Nathan <bossartn(at)amazon(dot)com> wrote:
>>> IIUC this would mean that archive modules that need to define GUCs or
>>> register background workers would have to separately define a
>>> _PG_init() and be loaded via shared_preload_libraries in addition to
>>> archive_library. That doesn't seem too terrible to me, but it was
>>> something I was trying to avoid.
>>
>> Hmm. That doesn't seem like a terrible goal, but I think we should try
>> to find some way of achieving it that looks tidier than this does.
>
> We could just treat archive_library as if it is tacked onto the
> shared_preload_libraries list. I think I can make that look
> relatively tidy.

Alright, here is an attempt at that. With this revision, archive
libraries are preloaded (and _PG_init() is called), and the archiver
is responsible for calling _PG_archive_module_init() to get the
callbacks. I've also removed the GUC check hooks as previously
discussed.

Nathan

Attachment Content-Type Size
v6-0001-Introduce-archive-module-framework.patch application/octet-stream 19.0 KB
v6-0002-Add-an-example-of-a-basic-archiving-module.patch application/octet-stream 12.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2021-10-25 20:02:34 Re: Assorted improvements in pg_dump
Previous Message Mark Dilger 2021-10-25 19:30:31 Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)