Paul Sheer wrote:
> Hadoop backend for PostGreSQL....
Resurrecting an old thread, it seems some guys at Yale implemented
something very similar to what this thread was discussing.
> >It's an open source stack that includes PostgreSQL Hadoop, and Hive, along
> >with some glue between PostgreSQL and Hadoop, a catalog, a data loader, and
> >an interface that accepts queries in MapReduce or SQL and generates query
> >plans that are processed partly in Hadoop and partly in different PostgreSQL
> >instances spread across many nodes in a shared-nothing cluster of machines.
Their detailed paper is here:
According to the paper, it scales very well.
> A problem that my client has, and one that I come across often,
> is that a database seems to always be associated with a particular
> physical machine, a physical machine that has to be upgraded,
> replaced, or otherwise maintained.
> Even if the database is replicated, it just means there are two or
> more machines. Replication is also a difficult thing to properly
> With a distributed data store, the data would become a logical
> object - no adding or removal of machines would affect the data.
> This is an ideal that would remove a tremendous maintenance
> burden from many sites ---- well, at least the one's I have worked
> at as far as I can see.
> Does anyone know of plans to implement PostGreSQL over Hadoop?
> Yahoo seems to be doing this:
> But they store tables column-ways for their performance situation.
> If one is doing a lot of inserts I don't think this is most efficient - ?
> Has Yahoo put the source code for their work online?
> Many thanks for any pointers.
In response to
pgsql-hackers by date
|Next:||From: Tom Lane||Date: 2009-07-22 03:35:30|
|Subject: Re: CommitFest 2009-07 - End of Week 1 |
|Previous:||From: Robert Haas||Date: 2009-07-22 03:18:42|
|Subject: CommitFest 2009-07 - End of Week 1|