Skip site navigation (1) Skip section navigation (2)

Re: Hadoop backend?

From: Hans-Jürgen Schönig <postgres(at)cybertec(dot)at>
To: "Jonah H(dot) Harris" <jonah(dot)harris(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, pi(dot)songs(at)gmail(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Hadoop backend?
Date: 2009-02-24 12:55:11
Message-ID: (view raw or whole thread)
Lists: pgsql-hackers
why not just stream it in via set-returning functions and make sure  
that we can mark a set returning function as "STREAMABLE" or so (to  
prevent joins, whatever).
is it the easiest way to get it right and it helps in many other cases.
i think that the storage manager is definitely the wrong place to do  

it is also easy to use more than just one backend then if you get the  
interface code right.


On Feb 24, 2009, at 12:03 AM, Jonah H. Harris wrote:

> On Sun, Feb 22, 2009 at 3:47 PM, Robert Haas <robertmhaas(at)gmail(dot)com>  
> wrote:
> In theory, I think you could make postgres work on any type of
> underlying storage you like by writing a second smgr implementation
> that would exist alongside md.c.  The fly in the ointment is that
> you'd need a more sophisticated implementation of this line of code,
> from smgropen:
>    reln->smgr_which = 0;   /* we only have md.c at present */
> I believe there is more than that which would need to be done  
> nowadays.  I seem to recall that the storage manager abstraction has  
> slowly been dedicated/optimized for md over the past 6 years or so.   
> It may even be easier/preferred to write a hadoop specific access  
> method depending on what you're looking for from hadoop.
> -- 
> Jonah H. Harris, Senior DBA

Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt

In response to

pgsql-hackers by date

Next:From: Peter EisentrautDate: 2009-02-24 13:08:32
Subject: Re: Hadoop backend?
Previous:From: Bruce MomjianDate: 2009-02-24 12:50:05
Subject: Re: Synchronous replication & Hot standby patches

Privacy Policy | About PostgreSQL
Copyright © 1996-2015 The PostgreSQL Global Development Group