Skip site navigation (1) Skip section navigation (2)

Re: Your review of pg_receivexlog/pg_basebackup

From: Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Subject: Re: Your review of pg_receivexlog/pg_basebackup
Date: 2011-10-26 07:52:58
Message-ID: 4EA7BC5A.7060103@enterprisedb.com (view raw or flat)
Thread:
Lists: pgsql-hackers
(CC'ing pgsql-hackers, this started as an IM discussion yesterday but 
really belongs in the archives)

On 25.10.2011 23:52, Magnus Hagander wrote:
>> There's a tiny chance to get incomplete xlog files with pg_receivexlog if you crash:
>> 1. pg_receivexlog finishes write()ing a file but system crashes before fsync() finishes.
>> 2. When pg_receivexlog restarts after crash, the last WAL file was not fully flushed to disk, with
>> holes in the middle, but it has the right length. pg_receivexlog will continue streaming from the next file.
>> not sure if we care about such a narrow window, but maybe we do
>
> So how would we go about fixing that?  Always unlink the last file in
> the directory and try from there would seem dangerous too - what if
> it's not available on the master anymore, then we might have given up
> on data...

Start streaming from the beginning of the last segment, but don't unlink 
it first. Just overwrite it as you receive the data.

Or, always create new xlog file as "0000000100000001000000D3.partial", 
and only when it's fully written, fsync it, and then rename it to 
"0000000100000001000000D3". Then you know that if a file doesn't have 
the .partial suffix, it's complete. The fact that the last partial file 
always has the .partial suffix needs some extra pushups in the 
restore_command, though.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

Responses

pgsql-hackers by date

Next:From: Shigeru HanadaDate: 2011-10-26 08:26:00
Subject: Re: pgsql_fdw, FDW for PostgreSQL server
Previous:From: Heikki LinnakangasDate: 2011-10-26 06:53:46
Subject: Re: Range Types - typo + NULL string constructor

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group