Skip site navigation (1) Skip section navigation (2)

Re: Streaming replication on win32, still broken

From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Streaming replication on win32, still broken
Date: 2010-02-17 06:03:44
Message-ID: 9837222c1002162203m4ef4139clc1d7b43c90e19948@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Wed, Feb 17, 2010 at 06:55, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> On Wed, Feb 17, 2010 at 6:28 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> If you send me your amazon id, I can get you premissions on my private
>> image. I plan to clean it up and make it public, just haven't gotten
>> around to it yet...
>
> Thanks for your concern! I'll send the ID when I complete the preparation.

ok.


> And, fortunately?, when I set wal_sync_method to open_sync, the problem was
> reproduced in the linux, too. The cause is that the data that is written by

Ah, that's good. It always helps if it's a cross-platform issue -
particularly in that it's not one of the funky win32 specific things
we did :)


> walreceiver is not aligned, even if O_DIRECT is used. On win32, O_DIRECT is
> used by default. So the problem always happened on win32.

Ahh. I see.


> I propose two solution ideas:
>
> 1. O_DIRECT is somewhat harmful in the standby since the data written by
>   walreceiver is read by the startup process immediately. So, how about
>   not making only walreceiver use O_DIRECT?

In that case, O_DIRECT would be counterproductive, no? It maps to
FILE_FLAG_NOI_BUFFERING, which makes sure it doesn't go into the
cache. So the read in the startup proc is actually guaranteed to
reuqire a physical read - of something we just wrote, so it'll almost
certainly end up waiting for a rotation, no?

Seems like getting rid of O_DIRECT here is the right thing to do,
regardless of this.


> 2. Straightforwardly observe the alignment rule. Since the received WAL
>   data might start at the middle of WAL block, walreceiver needs to keep
>   the last half-written WAL block for alignment. OTOH since the received
>   data might end at the middle of WAL block, walreceiver needs zero-padding.
>   As a result, walreceiver writes the set of the last WAL block, received
>   data and zero-padding.

May there be other reasons to d this as well?


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

In response to

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2010-02-17 06:03:52
Subject: Re: auto_explain causes regression failures
Previous:From: Fujii MasaoDate: 2010-02-17 05:55:41
Subject: Re: Streaming replication on win32, still broken

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group