Skip site navigation (1) Skip section navigation (2)

Re: Archival API

From: raymond(dot)siebert(at)mobilcom(dot)de
To: <simon(at)2ndquadrant(dot)com>
Cc: pgsql-hackers-pitr(at)postgresql(dot)org
Subject: Re: Archival API
Date: 2004-02-26 10:54:03
Message-ID: OF3F4D8087.7AF8F6C1-ONC1256E46.00373007-C1256E46.003BE191@mobilcom.de (view raw or flat)
Thread:
Lists: pgsql-hackers-pitr
Simon,

>So in summary, the API is:
>
>Archiver initialises and waits on notify
>Postgresql initialises
>...then
>Postgresql fills log, switches and close it, then calls
>XLogArchiveNotify()
>Archiver moves log somewhere safe, then sets state such that...
>...sometime later
>Postgresql checks XLogArchiveBusy() to see if its safe to recycle file
>and discovers the state set by 
>
>API is completely unintrusive on current tried and tested operation, and
>leaves the archiver(s) free to act as they choose, outside of the
>address space of PostgreSQL. That way we don't have to update regession
>tests with some destructive non-manual crash tests to show that works.
>
>Clearly, we wouldn't want WAL logs to hang around too long, so we need
>an initiation method for the archival process. Otherwise, we'll be
>writing "nnn.full" notifications yet without anybody ever deleting them.
>Either this could be set at startup with an archive_log_mode parameter
>(OK, the names been used before, but if the cap fits, wear it) or
>setting a maximum limit to number of archive logs and a few other ideas,
>none of which I like.

Another idea is an event handling system in database system. 
Sending defined signals to the 'outer world' through an API to a user 
supplied program.
This user supplied program can later on handle other events in addition to 
backup of WAL logs.
A fixed classification of events and a defined framework in user supplied 
program is the way to success.

First event can be 'WAL log archived' starting backup of an archived WAL 
log.
After archiver process has created an archived WAL log an event like 'WAL 
log archived' calls the user supplied program interpreting the event.
The archiver process must have a naming convention for archived WAL logs - 
especially for recovery.
Archiving WAL logs may require an archive destination - also needed for 
recovery.

Essential input to user supplied program is the full-pathed archived WAL 
log. 
A user supplied backup program backups full-pathed archived WAL log to an 
whatever Backup Destination (even /dev/null for suicide characters).
After successfull backup the archived WAL log needs to be removed.
All is handled internally in the user supplied program.

Advantage of this idea is that backup of archived WAL logs is 
automatically enabled without the manual work of an DB Admin person.

I have to admit I'm not a programer. How much effort to whole source code 
my idea will cost I can't calculate.

Any ideas ?

Raymond

Responses

pgsql-hackers-pitr by date

Next:From: Nicolai TufarDate: 2004-02-26 21:08:32
Subject: Re: Archival API
Previous:From: Simon RiggsDate: 2004-02-17 22:41:17
Subject: Archival API

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group