Skip site navigation (1) Skip section navigation (2)

Re: Hadoop backend?

From: Hans-Jürgen Schönig <postgres(at)cybertec(dot)at>
To: "Jonah H(dot) Harris" <jonah(dot)harris(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, pi(dot)songs(at)gmail(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Hadoop backend?
Date: 2009-02-24 12:55:11
Message-ID: D62A3AB4-243C-4721-8F46-E9EA0F319CC4@cybertec.at (view raw or flat)
Thread:
Lists: pgsql-hackers
why not just stream it in via set-returning functions and make sure  
that we can mark a set returning function as "STREAMABLE" or so (to  
prevent joins, whatever).
is it the easiest way to get it right and it helps in many other cases.
i think that the storage manager is definitely the wrong place to do  
this.

it is also easy to use more than just one backend then if you get the  
interface code right.

	regards,
	
		hans


On Feb 24, 2009, at 12:03 AM, Jonah H. Harris wrote:

> On Sun, Feb 22, 2009 at 3:47 PM, Robert Haas <robertmhaas(at)gmail(dot)com>  
> wrote:
> In theory, I think you could make postgres work on any type of
> underlying storage you like by writing a second smgr implementation
> that would exist alongside md.c.  The fly in the ointment is that
> you'd need a more sophisticated implementation of this line of code,
> from smgropen:
>
>    reln->smgr_which = 0;   /* we only have md.c at present */
>
> I believe there is more than that which would need to be done  
> nowadays.  I seem to recall that the storage manager abstraction has  
> slowly been dedicated/optimized for md over the past 6 years or so.   
> It may even be easier/preferred to write a hadoop specific access  
> method depending on what you're looking for from hadoop.
>
> -- 
> Jonah H. Harris, Senior DBA
> myYearbook.com
>


--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: www.postgresql-support.de

In response to

pgsql-hackers by date

Next:From: Peter EisentrautDate: 2009-02-24 13:08:32
Subject: Re: Hadoop backend?
Previous:From: Bruce MomjianDate: 2009-02-24 12:50:05
Subject: Re: Synchronous replication & Hot standby patches

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group