From: | Magnus Hagander <magnus(at)hagander(dot)net> |
---|---|
To: | Fujii Masao <masao(dot)fujii(at)gmail(dot)com> |
Cc: | Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>, Jaime Casanova <jaime(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Updated version of pg_receivexlog |
Date: | 2011-10-27 08:18:48 |
Message-ID: | CABUevEzP1NocR1X+GOgS+X6bA4xaYtd7rRjHqWC3xOsGqSzRKA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Thu, Oct 27, 2011 at 10:12, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> On Thu, Oct 27, 2011 at 4:49 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> On Thu, Oct 27, 2011 at 09:46, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>>> On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>>> On Thu, Oct 27, 2011 at 09:29, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>>>>> On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>>>>> I've applied this version with a few more minor changes that Heikki found.
>>>>>
>>>>> Cool!
>>>>>
>>>>> When I tried pg_receivexlog and checked the contents of streamed WAL file by
>>>>> xlogdump, I found that recent WAL records that walsender has already sent don't
>>>>> exist in that WAL file. I expected that pg_receivexlog writes the streamed WAL
>>>>> records to the disk as soon as possible, but it doesn't. Is this
>>>>> intentional? Or bug?
>>>>> Am I missing something?
>>>>
>>>> It writes it to disk as soon as possible, but doesn't fsync() until
>>>> the end of each segment. Are you by any chance looking at the file
>>>> while it's running?
>>>
>>> No. I looked at that file after shutting down the master server.
>>
>> Ugh, in that case something is certainly wrong. There is nothing but
>> setting up some offset values between PQgetCopyData() and write()...
>
> When end-of-copy stream is found or an error happens, pg_receivexlog
> exits without flushing outstanding WAL records. Which seems to cause
> the problem I reported.
Not sure I follow. When we arrive at PQgetCopyData() there should be
nothing buffered, and if the end of stream happens there it returns
-1, and we exit, no? So where is the data that's lost?
I do realize we don't actually fsync() and close() in this case - is
that what you are referring to? But the data should already have been
write()d, so it should still be there, no?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
From | Date | Subject | |
---|---|---|---|
Next Message | Fujii Masao | 2011-10-27 09:25:59 | Re: Updated version of pg_receivexlog |
Previous Message | Heikki Linnakangas | 2011-10-27 08:18:06 | Re: Hot Backup with rsync fails at pg_clog if under load |