Re: parallelizing the archiver

From: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
To: Julien Rouhaud <rjuju123(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, "Bossart, Nathan" <bossartn(at)amazon(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: parallelizing the archiver
Date: 2021-09-10 15:55:21
Message-ID: D0664258-CD16-4D8D-A4E2-BA72AE593F99@yandex-team.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> 10 сент. 2021 г., в 19:19, Julien Rouhaud <rjuju123(at)gmail(dot)com> написал(а):
> Wouldn't it be better to
> have a new archive_mode, e.g. "daemon", and have postgres responsible
> to (re)start it, and pass information through the daemon's
> stdin/stdout or something like that?

We don't even need to introduce new archive_mode.
Currently archive_command has no expectations regarding stdin\stdout.
Let's just say that we will push new WAL names to stdin until archive_command exits.
And if archive_command prints something to stdout we will interpret it as archived WAL names.
That's it.

Existing archive_commands will continue as is.

Currently information about what is archived is stored on filesystem in archive_status dir. We do not need to change anything.
If archive_command exits (with any exit code) we will restart it if there are WAL files that still were not archived.

Best regards, Andrey Borodin.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Mark Dilger 2021-09-10 16:20:38 Re: [Patch] ALTER SYSTEM READ ONLY
Previous Message Julien Rouhaud 2021-09-10 15:48:54 Re: parallelizing the archiver