Skip site navigation (1) Skip section navigation (2)

Streaming a base backup from master

From: Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Cc: Magnus Hagander <magnus(at)hagander(dot)net>
Subject: Streaming a base backup from master
Date: 2010-09-03 11:19:20
Message-ID: 4C80D9B8.2020301@enterprisedb.com (view raw or flat)
Thread:
Lists: pgsql-hackers
It's been discussed before that it would be cool if you could stream a 
new base backup from the master server, via libpq. That way you would 
not need low-level filesystem access to initialize a new standby.

Magnus mentioned today that he started hacking on that, and 
coincidentally I just started experimenting with it yesterday as well 
:-). So let's get this out on the mailing list.

Here's a WIP patch. It adds a new "TAKE_BACKUP" command to the 
replication command set. Upon receiving that command, the master starts 
a COPY, and streams a tarred copy of the data directory to the client. 
The patch includes a simple command-line tool, pg_streambackup, to 
connect to a server and request a backup that you can then redirect to a 
.tar file or pipe to "tar x".

TODO:

* We need a smarter way to do pg_start/stop_backup() with this. At the 
moment, you can only have one backup running at a time, but we shouldn't 
have that limitation with this built-in mechanism.

* The streamed backup archive should contain all the necessary WAL files 
too, so that you don't need to set up archiving to use this. You could 
just point the tiny client tool to the server, and get a backup archive 
containing everything that's necessary to restore correctly.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

Attachment: basebackup-1.patch
Description: text/x-diff (14.5 KB)

Responses

pgsql-hackers by date

Next:From: Thom BrownDate: 2010-09-03 11:25:35
Subject: Re: Streaming a base backup from master
Previous:From: Heikki LinnakangasDate: 2010-09-03 11:00:31
Subject: Re: thousand unrelated data files in pg_default tablespace

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group