Re: [RFC] Incremental backup v3: incremental PoC

From: Marco Nenciarini <marco(dot)nenciarini(at)2ndquadrant(dot)it>
To:
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [RFC] Incremental backup v3: incremental PoC
Date: 2015-01-16 16:55:42
Message-ID: 54B9428E.9020001@2ndquadrant.it
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 14/01/15 17:22, Gabriele Bartolini wrote:
>
> My opinion, Marco, is that for version 5 of this patch, you:
>
> 1) update the information on the wiki (it is outdated - I know you have
> been busy with LSN map optimisation)

Done.

> 2) modify pg_basebackup in order to accept a directory (or tar file) and
> automatically detect the LSN from the backup profile

New version of patch attached. The -I parameter now requires a backup
profile from a previous backup. I've added a sanity check that forbid
incremental file level backups if the base timeline is different from
the current one.

> 3) add the documentation regarding the backup profile and pg_basebackup
>

Next on my TODO list.

> Once we have all of this, we can continue trying the patch. Some
> unexplored paths are:
>
> * tablespace usage

I've improved my pg_restorebackup python PoC. It now supports tablespaces.

> * tar format
> * performance impact (in both "read-only" and heavily updated contexts)

From the server point of view, the current code generates a load similar
to normal backup. It only adds an initial scan of any data file to
decide whether it has to send it. One it found a single newer page it
immediately stop scanning and start sending the file. The IO impact
should not be that big due to the filesystem cache, but I agree with you
that it has to be measured.

Regards,
Marco

--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco(dot)nenciarini(at)2ndQuadrant(dot)it | www.2ndQuadrant.it

Attachment Content-Type Size
file-based-incremental-backup-v5.patch text/plain 34.3 KB
pg_restorebackup.py text/x-python-script 2.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jim Nasby 2015-01-16 16:57:07 Re: proposal: searching in array function - array_position
Previous Message Andres Freund 2015-01-16 16:50:37 Re: speedup tidbitmap patch: cache page