Re: Hadoop backend?

From: Hans-Jürgen Schönig <postgres(at)cybertec(dot)at>
To: "Jonah H(dot) Harris" <jonah(dot)harris(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, pi(dot)songs(at)gmail(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Hadoop backend?
Date: 2009-02-24 12:55:11
Message-ID: D62A3AB4-243C-4721-8F46-E9EA0F319CC4@cybertec.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

why not just stream it in via set-returning functions and make sure
that we can mark a set returning function as "STREAMABLE" or so (to
prevent joins, whatever).
is it the easiest way to get it right and it helps in many other cases.
i think that the storage manager is definitely the wrong place to do
this.

it is also easy to use more than just one backend then if you get the
interface code right.

regards,

hans

On Feb 24, 2009, at 12:03 AM, Jonah H. Harris wrote:

> On Sun, Feb 22, 2009 at 3:47 PM, Robert Haas <robertmhaas(at)gmail(dot)com>
> wrote:
> In theory, I think you could make postgres work on any type of
> underlying storage you like by writing a second smgr implementation
> that would exist alongside md.c. The fly in the ointment is that
> you'd need a more sophisticated implementation of this line of code,
> from smgropen:
>
> reln->smgr_which = 0; /* we only have md.c at present */
>
> I believe there is more than that which would need to be done
> nowadays. I seem to recall that the storage manager abstraction has
> slowly been dedicated/optimized for md over the past 6 years or so.
> It may even be easier/preferred to write a hadoop specific access
> method depending on what you're looking for from hadoop.
>
> --
> Jonah H. Harris, Senior DBA
> myYearbook.com
>

--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: www.postgresql-support.de

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2009-02-24 13:08:32 Re: Hadoop backend?
Previous Message Bruce Momjian 2009-02-24 12:50:05 Re: Synchronous replication & Hot standby patches