Skip site navigation (1) Skip section navigation (2)

Re: Updated version of pg_receivexlog

From: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>, Jaime Casanova <jaime(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Updated version of pg_receivexlog
Date: 2011-10-28 06:56:46
Message-ID: CAHGQGwGUT=bPb2VXyy_VbCMMdr-jm-FJeeW4gQ8gj+swvgQCVQ@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Thu, Oct 27, 2011 at 11:57 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Thu, Oct 27, 2011 at 16:54, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas
>>> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>>>> On 27.10.2011 14:09, Fujii Masao wrote:
>>>>> Yes. But that sounds unuserfriendly. Padding the WAL file manually
>>>>> is easy-to-do for a user?
>>
>>> I'd definitely want to avoid anything that requires pg_receivexlog to
>>> actually *parse* the WAL. That'll make it way more complex than I'd
>>> like.
>>
>> What parsing?  Just pad to 16MB with zeroes.  In fact, I think the
>
> I'm just sayihng that *if* parsing is required, it would be bad.
>
>> receiver should just create the file that size to start with, and then
>> write received data into it, much like normal WAL creation does.
>
> So when pg_receivexlog starts up, how would it know if the last file
> represents a completed file, or a half-full file, without actually
> parsing it? It could be a 16Mb file with 10 bytes of valid data, or a
> complete file with 16Mb of valid data.
>
> We could always ask for a retransmit of the whole file, but if that
> file is gone on the master, we won't be able to do that, and will
> error out in a situation that's not actually an error.
>
> Though I guess if we leave the file as .partial up until this point
> (per my other patch just posted), I guess this doesn't actually apply
> - if the file is called .partial, we'll overwrite into it. If it's
> not, then we assume it's a complete segment.

Yeah, I think that we should commit the patch that you posted in
other thread, and should change pg_receivexlog so that it creates
new WAL file filled up with zero or opens a pre-existing one, like
XLogFileInit() does, before writing any streamed data. If we do
this, a user can easily use a partial WAL file for recovery by
renaming that file.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

In response to

pgsql-hackers by date

Next:From: Cédric VillemainDate: 2011-10-28 08:40:33
Subject: Re: Add socket dir to pg_config..?
Previous:From: Fujii MasaoDate: 2011-10-28 06:46:49
Subject: Re: Your review of pg_receivexlog/pg_basebackup

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group