Skip site navigation (1) Skip section navigation (2)

Re: pipe chunking vs Windows

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pipe chunking vs Windows
Date: 2007-07-29 16:35:58
Message-ID: 46ACC1EE.8050002@dunslane.net (view raw or flat)
Thread:
Lists: pgsql-hackers

Magnus Hagander wrote:
> Andrew Dunstan wrote:
>   
>>>> I have just discovered that the recently implemented pipe chunking
>>>> protocol is broken on Windows. This is because the pipes are operating
>>>> in text mode and doing LF->CR-LF translation, so the number of bytes
>>>> received is not the number transmitted and set in the protocol header.
>>>>
>>>> I have not yet succeeded in turning this behaviour off (_setmode()
>>>> didn't seem to affect it). If we can't find a way to turn it off, the
>>>> only solution short of abandoning its use on Windows that I can think of
>>>> is to translate LF on input to something unlikely like 0x1C and then
>>>> translate it back when we read it from the pipe.
>>>>     
>>>>         
>>> At what point does it actually do the translation? Meaning what
>>> system/library call has it?
>>>
>>> Are we using the pipes from src/port/pipe.c? It does sound a bit weird
>>> that they'd do that, since it's basically just emulating stuff over
>>> standard tcp sockets, but perhaps something is broken in that code?
>>>
>>> Sorry, haven't really checked up on the chunk code yet, so I don't know
>>> offhand where to look.
>>>
>>>
>>>   
>>>       
>> It looks like we aren't. In fact. it looks like the only call to
>> pgpipe() in the whole source tree is in the syslogger and it's in
>> specifically non-Windows code, meaning that that whole file is currently
>> useless.
>>     
>
> Uh, see port.h, lines 212-224. If you're using the pipe() command to
> create it, it's used.
>   

No, it's the other way around :-) If you use pgpipe() on Unix you're 
calling pipe():


#ifndef WIN32
/*
 *  The function prototypes are not supplied because every C file
 *  includes this file.
 */
#define pgpipe(a)           pipe(a)
#define piperead(a,b,c)     read(a,b,c)
#define pipewrite(a,b,c)    write(a,b,c)
#else
extern int  pgpipe(int handles[2]);
extern int  piperead(int s, char *buf, int len);

#define pipewrite(a,b,c)    send(a,b,c,0)

#define PG_SIGNAL_COUNT 32
#define kill(pid,sig)   pgkill(pid,sig)
extern int  pgkill(int pid, int sig);
#endif


>
>   
>> Maybe you should have a good look at src/backend/postmaster/syslogger.c.
>> If we could get rid of the pipe-read threads and all the special Windows
>> cruft there that would certainly be an advance.
>>     
>
> I'll try to squeeze some time in to do that - I'll have to read up on
> the whole pipe/chunk thing first though, so it'll be a while.
>
>   

You don't need to understand the protocol (it's a very simple 
packetising protocol). The important point is that we have an anonymous 
pipe (created with CreatePipe) which has been dup'ed into stderr.

cheers

andrew

In response to

Responses

pgsql-hackers by date

Next:From: Magnus HaganderDate: 2007-07-29 16:39:17
Subject: Re: pipe chunking vs Windows
Previous:From: Magnus HaganderDate: 2007-07-29 16:08:40
Subject: Re: pipe chunking vs Windows

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group