Skip site navigation (1) Skip section navigation (2)

Re: pipe chunking vs Windows

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: "korry(dot)douglas" <korry(dot)douglas(at)enterprisedb(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pipe chunking vs Windows
Date: 2007-07-29 21:46:03
Message-ID: 46AD0A9B.2000401@dunslane.net (view raw or flat)
Thread:
Lists: pgsql-hackers

korry.douglas wrote:
>
>>> I have not yet succeeded in turning this behaviour off (_setmode() 
>>> didn't seem to affect it). If we can't find a way to turn it off, 
>>> the only solution short of abandoning its use on Windows that I can 
>>> think of is to translate LF on input to something unlikely like 0x1C 
>>> and then translate it back when we read it from the pipe.
>> Did you _setmode() the pipe, _setmode() stderr, or both?  (And did 
>> you try before or after calling dup2()?).
>>
>> It looks like the Win CRT implementation of dup2() copues the "mode" 
>> from the pipe that you've created into stderr.
> Sorry, I was looking at the wrong chunk of code in syslogger.c.
> Why are you calling _open_osfhandle() with O_TEXT?  That looks 
> suspicious giving the problem that you are seeing with CR -> CR/LF 
> translation.
>
>        

I have no idea why that's done - it goes back to the origins of the 
syslogger - probably because someone mistakenly thinks all WIndows text 
files have to have CRLF line endings.

I tried changing that to _O_BINARY, and calling _setmode on both the 
pipe before it's duped into stderr and stderr after the dup and both. 
Nothing seemed to work.

But that's not the only problem. I am now getting log file corruption 
even when I work around the text mode problem by not sending a '\n' at 
all, which makes me think even small packets aren't safely written 
atomically to Windows pipes. I wonder if we'd do better using the (so 
far unused) pipe simulation using sockets that we have. Not sure if we 
can dup a socket into stderr on Windows, but it might be worth trying, 
or even working around that problem. If we could safely use that the 
code would get a whole lot simpler - presumably we would no longer need 
those extra syslogger threads on Windows.

cheers

andrew

In response to

Responses

pgsql-hackers by date

Next:From: Andreas PflugDate: 2007-07-29 22:09:02
Subject: Re: pipe chunking vs Windows
Previous:From: korry.douglasDate: 2007-07-29 21:06:56
Subject: Re: pipe chunking vs Windows

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group