Skip site navigation (1) Skip section navigation (2)

Re: Hardware/OS recommendations for large databases ( 5TB)

From: "James Mello" <james(at)haydrian(dot)com>
To: "William Yu" <wyu(at)talisys(dot)com>,<pgsql-performance(at)postgresql(dot)org>
Subject: Re: Hardware/OS recommendations for large databases ( 5TB)
Date: 2005-11-15 19:07:40
Message-ID: 775F825B46B551499DC75DC4E92F80AB0E3623@jupiter.ad.haydrian.com (view raw or flat)
Thread:
Lists: pgsql-performance
Unless there was a way to guarantee consistency, it would be hard at
best to make this work. Convergence on large data sets across boxes is
non-trivial, and diffing databases is difficult at best. Unless there
was some form of automated way to ensure consistency, going 8 ways into
separate boxes is *very* hard. I do suppose that if you have fancy
storage (EMC, Hitachi) you could do BCV or Shadow copies. But in terms
of commodity stuff, I'd have to agree with Merlin. 

-----Original Message-----
From: pgsql-performance-owner(at)postgresql(dot)org
[mailto:pgsql-performance-owner(at)postgresql(dot)org] On Behalf Of William Yu
Sent: Tuesday, November 15, 2005 10:57 AM
To: pgsql-performance(at)postgresql(dot)org
Subject: Re: [PERFORM] Hardware/OS recommendations for large databases (
5TB)

Merlin Moncure wrote:
>>You could instead buy 8 machines that total 16 cores, 128GB RAM and
>  
> It's hard to say what would be better.  My gut says the 5u box would 
> be a lot better at handling high cpu/high concurrency problems...like 
> your typical business erp backend.  This is pure speculation of 
> course...I'll defer to the experts here.

In this specific case (data warehouse app), multiple machines is the
better bet. Load data on 1 machine, copy to other servers and then use a
middleman to spread out SQL statements to each machine.

I was going to suggest pgpool as the middleman but I believe it's
limited to 2 machines max at this time. I suppose you could daisy chain
pgpools running on every machine.

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
       choose an index scan if your joining column's datatypes do not
       match

Responses

pgsql-performance by date

Next:From: Bill McGonigleDate: 2005-11-15 19:12:23
Subject: Too Many OR's?
Previous:From: William YuDate: 2005-11-15 18:57:05
Subject: Re: Hardware/OS recommendations for large databases ( 5TB)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group