Re: Prevent restored WAL files from being archived again Re: Unnecessary WAL archiving after failover

On Mon, Jul 30, 2012 at 12:01 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> On Tue, Jun 5, 2012 at 3:37 PM, Noah Misch <noah(at)leadboat(dot)com> wrote:
>> On Fri, Mar 23, 2012 at 11:03:27PM +0900, Fujii Masao wrote:
>>> (2) WAL files which were restored from the archive
>>>
>>> In 9.1 or before, the restored WAL files don't remain after failover
>>> because they are always restored onto the temporary filename
>>> "RECOVERYXLOG". So the issue which I explain from now doesn't exist
>>> in 9.1 or before.
>>>
>>> In 9.2dev, as the result of supporting cascade replication,
>>> an archived WAL file is restored onto correct file name so that
>>> cascading walsender can send it to another standby. This restored
>>
>> The documentation still says this:
>>
>> WAL segments that cannot be found in the archive will be sought in pg_xlog/;
>> this allows use of recent un-archived segments. However, segments that are
>> available from the archive will be used in preference to files in
>> pg_xlog/. The system will not overwrite the existing contents of pg_xlog/
>> when retrieving archived files.
>>
>> I gather the last sentence is now false?
>
> Yes. Attached patch removes that sentence.
>
>>> WAL file has neither .ready nor .done archive status file. After
>>> failover, checkpoint checks the archive status file of the restored
>>> WAL file to attempt to recycle it, finds that it has neither .ready
>>> nor ,done, and creates .ready. Because of existence of .ready,
>>> it will be archived again even though it obviously already exists in
>>> the archival storage :(
>>>
>>> To prevent a restored WAL file from being archived again, I think
>>> that .done should be created whenever WAL file is successfully
>>> restored (of course this should happen only when archive_mode is
>>> enabled). Thought?
>>
>> Your proposed fix makes sense, and I cannot think of any disadvantage.
>> Concerning only doing it when archive_mode=on, would there ever be a case
>> where a segment is restored under archive_mode=off, then the server restarted
>> with archive_mode=on and an archival attempted on that segment?
>
> Yes, .done file should be created even if archive mode is not enabled.
>
> Attached patch changes the startup process so that it creates .done file
> whenever WAL file is successfully restored, whether archive mode is
> enabled or not. The restored WAL files will not be archived again because
> of .done file.

This patch can be applied cleanly for HEAD, but not in REL9_2_STABLE.
So here is the patch for REL9_2_STABLE.

Regards,

--
Fujii Masao