Skip site navigation (1) Skip section navigation (2)

Allow pg_archivecleanup to ignore extensions

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Allow pg_archivecleanup to ignore extensions
Date: 2011-02-08 07:57:42
Message-ID: 4D50F776.6080701@2ndquadrant.com (view raw or flat)
Thread:
Lists: pgsql-hackers
One bit of feedback I keep getting from people who archive their WAL 
files is that the fairly new pg_archivecleanup utility doesn't handle 
the case where those archives are compressed.  As the sort of users who 
are concerned about compression are also often ones with giant archives 
they struggle to cleanup, they would certainly appreciate having a 
bundled utility to take care of that. 

The attached patch provides an additional option to the utility to 
provide this capability.  It just strips a provided extension off any 
matching file it considers before running the test for whether it should 
be deleted or not.  It includes updates to the usage message and some 
docs about how this might be used.  Code by Jaime Casanova and myself.

Here's an example of it working:

$ psql -c "show archive_command"
                  archive_command                  
----------------------------------------------------
 cp -i %p archive/%f < /dev/null && gzip archive/%f
[Yes, I know that can be written more cleanly.  I call external scripts 
with more serious error handling than you can put into a single command 
line for this sort of thing in production.]

$ psql -c "select pg_start_backup('test',true)"
$ psql -c "select pg_stop_backup()"
$ psql -c "checkpoint"
$ psql -c "select pg_switch_xlog()"

$ cd $PGDATA/archive
$ ls
000000010000000000000025.gz
000000010000000000000026.gz
000000010000000000000027.gz
000000010000000000000028.00000020.backup.gz
000000010000000000000028.gz
000000010000000000000029.gz

$ pg_archivecleanup -d -x .gz `pwd`  
000000010000000000000028.00000020.backup
pg_archivecleanup: keep WAL file 
"/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000028" 
and later
pg_archivecleanup: removing file 
"/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000025.gz"
pg_archivecleanup: removing file 
"/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000027.gz"
pg_archivecleanup: removing file 
"/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000026.gz"
$ ls
000000010000000000000028.00000020.backup.gz
000000010000000000000028.gz
000000010000000000000029.gz

We recenty got some on-list griping that pg_standby doesn't handle 
archive files that are compressed, either.  Given how the job I'm 
working on this week is going, I'll probably have to add that feature 
next.  That's actually an easier source code hack than this one, because 
of how the pg_standby code modularizes the concept of a restore command.

-- 
Greg Smith   2ndQuadrant US    greg(at)2ndQuadrant(dot)com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us



Attachment: archivecleanup_extensions-v2.patch
Description: text/x-diff (5.2 KB)

Responses

pgsql-hackers by date

Next:From: Kevin GrittnerDate: 2011-02-08 08:43:06
Subject: Re: SSI patch version 14
Previous:From: Nick RudnickDate: 2011-02-08 07:23:15
Subject: Re: [pgsql-general 2011-1-21:] Are there any projects interested in object functionality? (+ rule bases)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group