Re: Sample archive_command is still problematic

Josh Berkus <josh(at)agliodbs(dot)com> wrote:

> The example archive_command we give in the docs is this one:
>
> archive_command = 'test ! -f /mnt/server/archivedir/%f && cp %p /mnt/server/archivedir/%f'
>
> This is a problematic recommendation.

I agree with that statement, ...

> If there's any reason why copying the archive file gets
> interrupted (storage blip, for example), then the command will
> fail and will continue to fail forever, ending archiving.

... but not for that reason.

> Is there some good reason why "test ! -f" was added to the
> sample?

In an environment with more than one cluster archiving, it is
otherwise way too easy to copy a config file and have the WAL files
of the two systems overwriting one another.  I consider a check for
an already existing file on the target to be very good practice.
The errors in the log are a clue that something went wrong, and
gives you a chance to fix things without data loss.

The problem with the recommended command is that cp is not atomic.
The file can be read before the contents are materialized, causing
early end to recovery.  I have seen it happen.  The right way to do
this is to copy to a different name or directory and mv the file
into place once it is complete -- or use software which does that
automatically, like rsync does.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company