Re: Would it be possible to have parallel archiving?

From: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: David Steele <david(at)pgmasters(dot)net>, hubert depesz lubaczewski <depesz(at)depesz(dot)com>, pgsql-hackers mailing list <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Would it be possible to have parallel archiving?
Date: 2018-08-28 20:01:21
Message-ID: 093DF1A3-05BC-45A4-B4C0-F62CD43657A8@yandex-team.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> 28 авг. 2018 г., в 14:08, Stephen Frost <sfrost(at)snowman(dot)net> написал(а):
>
> Greetings,
>
> * David Steele (david(at)pgmasters(dot)net <mailto:david(at)pgmasters(dot)net>) wrote:
>> On 8/28/18 8:32 AM, Stephen Frost wrote:
>> To be clear, pgBackRest uses the .ready files in archive_status to
>> parallelize archiving but still notifies PostgreSQL of completion via
>> the archive_command mechanism. We do not modify .ready files to .done
>> directly.
>
> Right, we don't recommend mucking around with that directory of files.
> Even if that works today (which you'd need to test extensively...),
> there's no guarantee that it'll work and do what you want in the
> future...
WAL-G modifies archive_status files.
This path was chosen to limit state preserved between WAL-G runs (archiving to S3) and further push archiving performance.
Indeed, it was very hard to test. Also, this makes impossible to use two archiving system simultaneously for transit period.

Best regards, Andrey Borodin.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Stephen Frost 2018-08-28 20:07:54 Re: Would it be possible to have parallel archiving?
Previous Message Tom Lane 2018-08-28 19:53:39 pg_dump --load-via-partition-root vs. parallel restore