Re: Proposal: Incremental Backup

From: Benedikt Grundmann <bgrundmann(at)janestreet(dot)com>
To: Gabriele Bartolini <gabriele(dot)bartolini(at)2ndquadrant(dot)it>
Cc: Marco Nenciarini <marco(dot)nenciarini(at)2ndquadrant(dot)it>, Bruce Momjian <bruce(at)momjian(dot)us>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Claudio Freire <klaussfreire(at)gmail(dot)com>, desmodemone <desmodemone(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Proposal: Incremental Backup
Date: 2014-08-08 06:57:23
Message-ID: CADbMkNMH6W6cGaL1=Zcvp5qurqTjUHuM838NbOJ4C=cZ58hJTw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Aug 7, 2014 at 6:29 PM, Gabriele Bartolini <
gabriele(dot)bartolini(at)2ndquadrant(dot)it> wrote:

> Hi Marco,
>
> > With the current full backup procedure they are backed up, so I think
> > that having them backed up with a rsync-like algorithm is what an user
> > would expect for an incremental backup.
>
> Exactly. I think a simple, flexible and robust method for file based
> incremental backup is all we need. I am confident it could be done for
> 9.5.
>
> I would like to quote every single word Simon said. Block level
> incremental backup (with Robert's proposal) is definitely the ultimate
> goal for effective and efficient physical backups. I see file level
> incremental backup as a very good "compromise", a sort of intermediate
> release which could nonetheless produce a lot of benefits to our user
> base, for years to come too.
>
> Thanks,
> Gabriele
>

I haven't been following this discussion closely at all. But at Janestreet
we have been using pg_start_backup together with rsync --link-dest (onto a
big NFS) to achieve incremental stored backup. In our experience this
works very well, it is however advisable to look into whatever is used to
serve the NFS as we had to set some options to increase the maximum number
of hardlinks.

Cheers,

Bene

>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Guillaume Lelarge 2014-08-08 07:08:50 Maximum number of WAL files in the pg_xlog directory
Previous Message Ashutosh Bapat 2014-08-08 06:27:51 Re: jsonb format is pessimal for toast compression