Re: Reduce the time required for a database recovery from archive.

From: David Steele <david(at)pgmasters(dot)net>
To: Dmitry Shulga <d(dot)shulga(at)postgrespro(dot)ru>, Stephen Frost <sfrost(at)snowman(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Reduce the time required for a database recovery from archive.
Date: 2021-03-18 15:04:19
Message-ID: dd7f8b17-0c87-3e2c-7a99-f972713dc227@pgmasters.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Dimtry,

On 1/11/21 2:51 AM, Dmitry Shulga wrote:
> Hi Stephen
>
> Based on our last discussion I redesigned the implementation of WAL
> archive recovery speed-up.

Seems like there should have been a patch attached? In any case the
current patch no longer applies so marked Waiting on Author.

Personally, I'm not too keen on this patch as implemented. Several
third-party backup solutions support parallel archive get so it would be
nice to support an interface that simply says to the restore_command,
"go get 1gb of WAL and write the files here." This patch still assumes
that the user has written their own restore command, which is
third-party by definition, so I can't see how interfacing with
third-party software is an issue here.

Also, having multiple workers blindly asking for WAL can cause quite a
bit of traffic and cost because PG knows what WAL it wants but it
doesn't know what exists. On the other hand, a backup solution can
cheaply determine what is available to prevent hammering the archive
with requests for files that don't exist.

Regards,
--
-David
david(at)pgmasters(dot)net

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message John Naylor 2021-03-18 15:09:34 Re: Force lookahead in COPY FROM parsing
Previous Message Pavel Stehule 2021-03-18 15:00:24 Re: pl/pgsql feature request: shorthand for argument and local variable references