Would it be possible to have parallel archiving?

From: hubert depesz lubaczewski <depesz(at)depesz(dot)com>
To: pgsql-hackers mailing list <pgsql-hackers(at)postgresql(dot)org>
Subject: Would it be possible to have parallel archiving?
Date: 2018-08-28 06:02:21
Message-ID: 20180828060221.x33gokifqi3csjj4@depesz.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,
I'm in a situation where we quite often generate more WAL than we can
archive. The thing is - archiving takes long(ish) time but it's
multi-step process and includes talking to remote servers over network.

I tested that simply by running archiving in parallel I can easily get
2-3 times higher throughput.

But - I'd prefer to keep postgresql knowing what is archived, and what
not, so I can't do the parallelization on my own.

So, the question is: is it technically possible to have parallel
archivization, and would anyone be willing to work on it (sorry, my
c skills are basically none, so I can't realistically hack it myself)

Best regards,

depesz

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alexander Kukushkin 2018-08-28 06:21:57 Re: BUG #15346: Replica fails to start after the crash
Previous Message Heikki Linnakangas 2018-08-28 05:45:50 Re: First steps to being a contributer