Re: Extending BASE_BACKUP in replication protocol: incremental backup and backup format

From: Jim Nasby <jim(at)nasby(dot)net>
To: Magnus Hagander <magnus(at)hagander(dot)net>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
Cc: Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>, PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Extending BASE_BACKUP in replication protocol: incremental backup and backup format
Date: 2014-01-15 06:46:52
Message-ID: 52D62EDC.9040301@nasby.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 1/14/14, 7:41 AM, Magnus Hagander wrote:
> Yes, it would be necessary to scan the whole database as the LSN to be
> checked is kept in PageHeaderData :). Perhaps it is not that
> performant, but my initial thought was that perhaps the amount of data
> necessary to maintain incremental backups could balance with the
> amount of WAL necessary to keep and limit the whole amount on disk.
>
>
> It wouldn't be worse performance wise than a full backup. That one also has to read all the blocks after all... You're decreasing network traffic and client storage, with the same I/O on the server side. Seems worthwhile.

If there's enough demand, it probably wouldn't be that hard to keep a copy of the page LSNs in a fork; you only need to ensure that the LSN in the fork must be older than the LSN on disk could possibly be, and you wouldn't have to update the fork every time.

BTW, an incremental backup could possibly be useful as a way to catch a streaming replica up that's fallen way behind. The write IO would be sequential instead of trying to random-write while processing each WAL record.
--
Jim C. Nasby, Data Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alexander Korotkov 2014-01-15 06:47:02 Re: GIN improvements part 1: additional information
Previous Message Erik Rijkers 2014-01-15 06:46:28 Re: tests for client programs