Re: Updated version of pg_receivexlog

On Thu, Oct 27, 2011 at 4:49 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Thu, Oct 27, 2011 at 09:46, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>> On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>> On Thu, Oct 27, 2011 at 09:29, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>>>> On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>>>> I've applied this version with a few more minor changes that Heikki found.
>>>>
>>>> Cool!
>>>>
>>>> When I tried pg_receivexlog and checked the contents of streamed WAL file by
>>>> xlogdump, I found that recent WAL records that walsender has already sent don't
>>>> exist in that WAL file. I expected that pg_receivexlog writes the streamed WAL
>>>> records to the disk as soon as possible, but it doesn't. Is this
>>>> intentional? Or bug?
>>>> Am I missing something?
>>>
>>> It writes it to disk as soon as possible, but doesn't fsync() until
>>> the end of each segment. Are you by any chance looking at the file
>>> while it's running?
>>
>> No. I looked at that file after shutting down the master server.
>
> Ugh, in that case something is certainly wrong. There is nothing but
> setting up some offset values between PQgetCopyData() and write()...

When end-of-copy stream is found or an error happens, pg_receivexlog
exits without flushing outstanding WAL records. Which seems to cause
the problem I reported.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center