Skip site navigation (1) Skip section navigation (2)

Re: POSIX shared memory support

From: Chris Marcellino <cmarcellino(at)apple(dot)com>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: pgsql-patches(at)postgresql(dot)org
Subject: Re: POSIX shared memory support
Date: 2007-02-27 09:09:46
Message-ID: 220335F2-1356-4309-ADBB-1EAD2CD2A19D@apple.com (view raw or flat)
Thread:
Lists: pgsql-patches
The Win32 version didn't materialize until very recently. The Win32  
calls are similar semantically to the POSIX ones, so it was somewhat  
straightforward.

Plaintext is nice if you can fit it, since Windows permits you to  
have slashes and all sorts of other non-filename characters in them,  
unlike POSIX shmem. Also for POSIX, certain platforms (ahem Darwin  
has 30 chars) have very small segment name limits. As Tom said, the  
renaming-while-running issue might be a deal breaker, but I'm not  
sure that is a problem on Windows. It sounds like you are on the  
right track.

I just mentioned this to Tom, but a solution for POSIX might be to  
use the device and inode combination for the data directory, since  
that is constant across renames and is more certain to be unique.  
Like replacing the GenerateIPCName() function with something this:

static void
GenerateIPCName(char destIPCName[IPCSegNameLength])
{
	struct		stat statbuf;
	
	/* Get the data directory's device and inode */
	if (stat(DataDir, &statbuf) < 0)
		ereport(FATAL,
				(errcode_for_file_access(),
				 errmsg("could not stat data directory \"%s\": %m",
						DataDir)));
	
	/*
	 * POSIX requires that shared memory names begin with a single slash.
	 * They should not have any others slashes or any non-alphanumerics to
	 * maintain the broadest assumption of what is permitted in a filename.
	 * Also, case sensitivity should not be presumed.
	 */
	snprintf(destIPCName, IPCSegNameLength, "/PostgreSQL.%jx.%jx",
			 (intmax_t) statbuf.st_dev, (intmax_t) statbuf.st_ino);
}


Does Windows have a method to get a unique ID number for a given data  
directory, or a token file in that directory? It would need to be  
constant while the database is open. Perhaps  
GetFileInformationByHandle? It returns a struct with a nFileIndex  
value that seems to be that, although I'm not certain.
This might make it easier to avoid the complexity of fitting the  
filename in the segment name, and avoid the rename problem,

Thanks,
Chris Marcellino




On Feb 27, 2007, at 12:56 AM, Magnus Hagander wrote:

> On Mon, Feb 26, 2007 at 09:00:09PM -0800, Chris Marcellino wrote:
>
>>
>> There is also a Windows version of this patch included, which can
>> replace the current SysV-to-Win32 shared memory
>> layer as it currently does not check for orphaned backends in the
>> database. If this is used,
>> src/backend/port/win32/shmem.c and its makefile reference can be
>> removed.
>
> This code is pretty close to the one I've been working on. Didn't
> reliase you would be working on a win32 specific version, thought you
> were doing POSIX only :-O
>
> Anyway. My version does not use hashing of the name, it just puts the
> name all the way in there. What are peoples feeling about that - win32
> supports 32768 characters (though those are UTF16, so maybe the real
> value is 16384 - still, if someone has his data directory that deep  
> down
> in a path, he deserves not to have it work). Should we be using  
> hashing
> or just plaintext there? I can see the argument for plaintext being  
> that
> you can then see in process explorer what data directory a backend is
> connected to. But I'm open to arguments as to why a hash would be  
> better
> :-)
>
>
> While looking at this, I noticed that Windows 2003 SP1 adds a  
> parameter
> SEC_LARGE_PAGES that enables "large pages to be used when mapping  
> images
> or backing from the pagefile". I would assume this is for performance
> ;-) Does anybody know anything more about this? Worth testing out  
> to see
> if it increases performance?
>
>
>> The patches are available here (the postings were being silently
>> dropped when I attached this large of a file):
>
> Actually, I don't think it's becauseo f the size of the file. Last  
> time
> I checked we still had a configuration which had the lists silently  
> drop
> certain file extensions, including .tar, .zip and .tar.gz. I've had
> several patches dropped that way. I've had no confirmation it has been
> fixed, so I'm assuming it's still a problem.
>
> //Magnus


In response to

Responses

pgsql-patches by date

Next:From: Magnus HaganderDate: 2007-02-27 09:30:15
Subject: Re: POSIX shared memory support
Previous:From: ITAGAKI TakahiroDate: 2007-02-27 08:56:23
Subject: Re: Dead Space Map version 2

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group