Re: Proposal: Incremental Backup

From: Claudio Freire <klaussfreire(at)gmail(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: Simon Riggs <simon(at)2ndquadrant(dot)com>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Gabriele Bartolini <gabriele(dot)bartolini(at)2ndquadrant(dot)it>, desmodemone <desmodemone(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Marco Nenciarini <marco(dot)nenciarini(at)2ndquadrant(dot)it>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Proposal: Incremental Backup
Date: 2014-08-06 16:15:32
Message-ID: CAGTBQpYpCyqCz_Ecd1w9T4X18fTjK68Qt+Wi5=WpEKr10=88eA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Aug 6, 2014 at 12:20 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
> Well, for file-level backups we have:
>
> 1) use file modtime (possibly inaccurate)
> 2) use file modtime and checksums (heavy read load)
>
> For block-level backups we have:
>
> 3) accumulate block numbers as WAL is written
> 4) read previous WAL at incremental backup time
> 5) read data page LSNs (high read load)
>
> The question is which of these do we want to implement? #1 is very easy
> to implement, but incremental _file_ backups are larger than block-level
> backups. If we have #5, would we ever want #2? If we have #3, would we
> ever want #4 or #5?

You may want to implement both #3 and #2. #3 would need a config
switch to enable updating the bitmap. That would make it optional to
incur the I/O cost of updating the bitmap. When the bitmap isn't
there, the backup would use #2. Slow, but effective. If slowness is a
problem for you, you enable the bitmap and do #3.

Sounds reasonable IMO, and it means you can start by implementing #2.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bill Epstein 2014-08-06 16:17:46 Questions on dynamic execution and sqlca
Previous Message Robert Haas 2014-08-06 16:02:11 Re: add modulo (%) operator to pgbench