Skip site navigation (1) Skip section navigation (2)

Re: [PERFORM] Postgres and really huge tables

From: Scott Marlowe <smarlowe(at)g2switchworks(dot)com>
To: Brian Hurt <bhurt(at)janestcapital(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org, pgsql-advocacy(at)postgresql(dot)org
Subject: Re: [PERFORM] Postgres and really huge tables
Date: 2007-01-18 21:04:07
Message-ID: 1169154247.9586.100.camel@state.g2switchworks.com (view raw or flat)
Thread:
Lists: pgsql-advocacypgsql-performance
On Thu, 2007-01-18 at 14:31, Brian Hurt wrote:
> Is there any experience with Postgresql and really huge tables?  I'm 
> talking about terabytes (plural) here in a single table.  Obviously the 
> table will be partitioned, and probably spread among several different 
> file systems.  Any other tricks I should know about?
> 
> We have a problem of that form here.  When I asked why postgres wasn't 
> being used, the opinion that postgres would "just <explicitive> die" was 
> given.  Personally, I'd bet money postgres could handle the problem (and 
> better than the ad-hoc solution we're currently using).  But I'd like a 
> couple of replies of the form "yeah, we do that here- no problem" to 
> wave around.

It really depends on what you're doing.

Are you updating every row by a single user every hour, or are you
updating dozens of rows by hundreds of users at the same time?

PostgreSQL probably wouldn't die, but it may well be that for certain
batch processing operations it's a poorer choice than awk/sed or perl.

If you do want to tackle it with PostgreSQL, you'll likely want to build
a truly fast drive subsystem.  Something like dozens to hundreds of
drives in a RAID-10 setup with battery backed cache, and a main server
with lots of memory on board.

But, really, it depends on what you're doing to the data.

In response to

pgsql-performance by date

Next:From: Arjen van der MeijdenDate: 2007-01-18 21:10:12
Subject: Re: Configuration Advice
Previous:From: Matthew T. O'ConnorDate: 2007-01-18 20:40:07
Subject: Re: Autoanalyze settings with zero scale factor

pgsql-advocacy by date

Next:From: Chris MairDate: 2007-01-18 21:42:40
Subject: Re: Postgres and really huge tables
Previous:From: Joshua D. DrakeDate: 2007-01-18 20:39:45
Subject: Re: Postgres and really huge tables

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group