Re: pg_archivecleanup should remove WAL files also in pg_xlog?

On 13.12.2010 09:50, Fujii Masao wrote:
> On Mon, Dec 13, 2010 at 4:28 PM, Heikki Linnakangas
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> On 13.12.2010 08:44, Fujii Masao wrote:
>>>
>>> pg_archivecleanup removes unnecessary WAL files from the archive, but not
>>> from pg_xlog directory. So, after failover, those WAL files might
>>> exist in pg_xlog and be archived again later.
>>
>> A file that has already been archived successfully should not be archived
>> again. The server keeps track of which files it has already archived with
>> the .ready/.done files.
>
> This seems to require
>
> * archiver to save the last archived WAL file name in the shmem
> * walsender to send it to walreceiver
> * walreceiver to create .done file when it's arrived
> * bgwriter not to remove WAL files which don't have .done file in standby
>
> Right? One good side effect of this is that we can prevent WAL files from
> being removed from the standby before the master archives them.

Oh, you said "after failover", I missed that. So the problem is that the
standby might try to re-archive files that the master already archived.

If the only consequence is that you get some extra WAL files in the
archive, until pg_archivecleanup runs again, I think we can just live
with it. But don't you have bigger problems when standby tries to
archive a file that already exists in the archive, because master
already archived it? We advise to write archive_command so that it fails
if the file exists already.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com