Skip site navigation (1) Skip section navigation (2)

Re: Planning a Large RDBMS

From: Jeff Davis <pgsql(at)j-davis(dot)com>
To: Alessandro Gagliardi <alessandro(at)path(dot)com>
Cc: pgsql-novice(at)postgresql(dot)org
Subject: Re: Planning a Large RDBMS
Date: 2012-06-17 06:56:22
Message-ID: 1339916182.15719.39.camel@jdavis (view raw or flat)
Thread:
Lists: pgsql-novice
On Fri, 2012-06-15 at 11:11 -0700, Alessandro Gagliardi wrote: 
> We've been using MongoDB for the past couple years and are talking
> about moving to a more traditional (i.e. stable) database system for
> OLTP. I've been using RDBMS for OLAP already, but the demands are
> somewhat different. But we figure, if Facebook can do it, we can. The
> expertise around here is more suited for MySQL, but I, for one, am
> fond of the features in Postgres that you can't find in MySQL. Our
> Mongo system is sharded across 11 logical nodes on Amazon's EC2. Our
> database is in the hundreds of gigabytes (i.e. too big for memory on
> one machine) and we can expect it to exceed a terabyte before too
> long. We have the distinct advantage of being able to plan this from
> the ground up. I've read a bit about partitioning and table spaces,
> but from what I can tell, those solutions still require the database
> to be hosted by a single machine which may not suffice for our
> purposes.

I suggest first identifying what portions of the data are largely
independent versus closely tied together. Then use that to determine how
it makes sense to lay these out across multiple machines. Then, just use
different instances of postgres across the different machines. You'll
need to think carefully about how you interact with those different
instances, but you've already done a lot of that thinking for your
current setup.

One way to think about it is that one postgres instance on one machine
is the unit of consistency.

Regards,
	Jeff Davis





In response to

pgsql-novice by date

Next:From: Joseph MarlinDate: 2012-06-18 13:29:24
Subject: Impatient warm standby - Recovery spam in pg_log/startup.log
Previous:From: Jeff DavisDate: 2012-06-17 01:52:36
Subject: Re: Problem creating trigger

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group