Re: Updated version of pg_receivexlog

On Thu, Oct 27, 2011 at 11:57 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Thu, Oct 27, 2011 at 16:54, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas
>>> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>>>> On 27.10.2011 14:09, Fujii Masao wrote:
>>>>> Yes. But that sounds unuserfriendly. Padding the WAL file manually
>>>>> is easy-to-do for a user?
>>
>>> I'd definitely want to avoid anything that requires pg_receivexlog to
>>> actually *parse* the WAL. That'll make it way more complex than I'd
>>> like.
>>
>> What parsing?  Just pad to 16MB with zeroes.  In fact, I think the
>
> I'm just sayihng that *if* parsing is required, it would be bad.
>
>> receiver should just create the file that size to start with, and then
>> write received data into it, much like normal WAL creation does.
>
> So when pg_receivexlog starts up, how would it know if the last file
> represents a completed file, or a half-full file, without actually
> parsing it? It could be a 16Mb file with 10 bytes of valid data, or a
> complete file with 16Mb of valid data.
>
> We could always ask for a retransmit of the whole file, but if that
> file is gone on the master, we won't be able to do that, and will
> error out in a situation that's not actually an error.
>
> Though I guess if we leave the file as .partial up until this point
> (per my other patch just posted), I guess this doesn't actually apply
> - if the file is called .partial, we'll overwrite into it. If it's
> not, then we assume it's a complete segment.

Yeah, I think that we should commit the patch that you posted in
other thread, and should change pg_receivexlog so that it creates
new WAL file filled up with zero or opens a pre-existing one, like
XLogFileInit() does, before writing any streamed data. If we do
this, a user can easily use a partial WAL file for recovery by
renaming that file.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center