Skip site navigation (1) Skip section navigation (2)

Re: Shared memory

From: Thomas Hallgren <thomas(at)tada(dot)se>
To: Martijn van Oosterhout <kleptog(at)svana(dot)org>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Shared memory
Date: 2006-03-24 13:29:37
Message-ID: 4423F441.10306@tada.se (view raw or flat)
Thread:
Lists: pgsql-hackers
Martijn van Oosterhout wrote:
> On Fri, Mar 24, 2006 at 11:51:30AM +0100, Thomas Hallgren wrote:
>   
>> Hi,
>> I'm currently investigating the feasibility of an alternative PL/Java 
>> implementation that would use shared memory to communicate between a JVM 
>> and the backend processes. I would very much like to make use of the 
>> routines provided in shmem.c but I'm a bit uncertain how to add a segment 
>> for my own use.
>>     
>
> I'm wondering if a better way to do it would be similar to the way X
> does it. The client connects to the X server via a pipe (tcp/ip or unix
> domain). This is handy because you can block on a pipe. The client then
> allocates a shared memory segment and sends a message to the server,
> who can then also connect to it.
>
> The neat thing about this is that the client can put data in the shared
> memory segment and send one byte through the pipe and then block on a
> read. The JVM which has a thread waiting on the other end wakes up,
> processes the data, puts the result back and writes a byte to the pipe
> and waits. This wakes up the client who can then read the result.
>
> No locking, no semaphores, the standard UNIX semantics on pipes and
> sockets make sure everything works.
>
> In practice you'd probably end up sending small responses exclusively
> via the pipe and only use the shared memory for larger blocks of data
> but that's your choice. In X this is mostly used for image data and
> such.
>
>   

Pipes could be used when the connection is initialized, that's for sure. 
Thanks for the suggestion. Only thing I need to solve is how to detect 
if the JVM is present and start it up when it isn't. Either I require 
that it's there and generate an error when it isn't (analog with what 
Apache would do if Tomcat is missing) or I treat the failure to obtain 
the pipe as an indication on that it's not started yet.



>> My questions are:
>> 1. Do you see something right away that invalidates this approach?
>>     
>
> Nothing direct, though a single segment just for finding the JVM seems
> a lot. A socket approach would work better I think.
>
>   
For the initial setup, sure. But I think pipes might be too slow for the 
actual function calls. What I want is the absolute most efficient ipc 
mechanism that can be achived. I'm thinking in terms of critical 
sections obtained using spinlocks and atomic exchange on memory that 
perhaps migrate to a real semaphore when the spin goes on for too long.

I will do some tests using pipes too. If the gain using other types of 
concurrency control is second to none, then I would agree that pipes are 
simpler and more elegant.

>> 2. Is using the shared memory functionality that the backend provides a 
>> good idea (I'm thinking shmem functions, critical sections, semaphores, 
>> etc.). I'd rather depend on them then having conditional code for different 
>> operating systems.
>>     
>
> That I don't know. However, ISTM a lock-free approach is better
> wherever possible. If you can avoid the semaphores altogether...
>
>   
Lock free? I'm not sure I understand what you mean. I'll have to wait on 
something. Or are you referring to the pipe approach?


>> 3. Would it be better if the Postmaster allocated the global segment and 
>> started the JVM (based on some config parameter)?
>>     
>
> I don't know about the segment but the postmaster should start. I
> thought the tsearch guys had an approach using a co-process. I don't
> know how they start it up but they connected via pipes.
>
>   
I'll check that out. Thanks for the tip.

> Hope this helps,
>   
Your insights often do. Thanks a lot.

Regards,
Thomas Hallgren



In response to

pgsql-hackers by date

Next:From: John DeSoiDate: 2006-03-24 13:54:50
Subject: Re: pgNixInstaller: Making PostgreSQL relocatable
Previous:From: Jim C. NasbyDate: 2006-03-24 13:21:44
Subject: Nightly builds (was: [SQL] Function Parameters in GROUP BY clause cause errors)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group