Re: Simplifying replication

Hannu Krosing <hannu(at)2ndQuadrant(dot)com> writes:
>> > To make pg_basebackup.py self-sufficient it should also open 2nd
>> > connection to the same master and make sure that all WAL files are
>> > copied for the duration of base copy.

Done now, please have a look and try it if possible:

https://github.com/dimitri/pg_basebackup

> btw, as next step you could backport this to 8.x and have most of the
> benefits of SR. It should not be very hard to keep track of wal position
> inside a pl/pythonu function and send one or more records back in form of

You now have the -x and -D options to set that up, but I didn't check
the backport part: it still depends on the pg_bb_list_files() function
to get the recursive listing of the pg_xlog directory, and it does that
using a WITH RECURSIVE query.

The way I did it is to only copy the (whole) WAL again if its ctime
changed since last loop. Also pg_basebackup won't start a backup if you
run it on its own, I don't think that's what you want here.

Oh, as I needed to fork() a process to care for the pg_xlog in a loop
while the base backup is sill ongoing, I added a -j --jobs option so
that you can hammer the master some more by having more than one process
doing the copying.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support