From: | elein <elein(at)varlena(dot)com> |
---|---|
To: | Joe Conway <mail(at)joeconway(dot)com> |
Cc: | elein <elein(at)varlena(dot)com>, pgsql-general <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: Large Databases |
Date: | 2004-08-31 21:30:22 |
Message-ID: | 20040831143022.F17217@cookie.varlena.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
I thought NFS was not recommended. Did I misunderstand this
or is there some kind of limitation to using different kinds(?)
of NFS.
Thank you for the excellent info.
--elein
On Tue, Aug 31, 2004 at 01:54:41PM -0700, Joe Conway wrote:
> elein wrote:
> >What is the linux and/or postgres limitation for very
> >large databases, if any? We are looking at 6T-20T.
> >My understanding is that if the hardware supports it,
> >then it can be done in postgres. But can hardware
> >support that?
>
> I've recently been going through a project to support what will become a
> 5 to 6 TB Postgres database (initially it will be about 300GB after
> conversion from the source system). A few significant things I've
> learned along the way:
>
> 1) The linux 2.4 kernel has a block device size limit of 2 TB.
>
> 2) The linux 2.6 kernel supports *huge* block device size -- I don't
> have it in front of me, but IIRC it was in the peta-bytes range.
>
> 3) xfs, jfs, and ext3 all can handle more than the 6TB we needed them to
> handle.
>
> 4) One of the leading SAN vendors initially claimed to be able to
> support our desire to have a single 6TB volume. We found that when
> pushed hard, we would get disk corruption (archives are down, but see
> HACKERS on 8/21/04 for a message I posted on the topic). Now we are
> being told that they don't support the linux 2.6 kernel, and
> therefore don't support > 2TB volumes.
>
> So the choices seem to be:
> a) Use symlinks or Postgres 8.0.0beta tablespaces to split your data
> across multiple 2 TB volumes.
>
> b) Use NFS mounted NAS.
>
> We are already a big NetApp shop, so NFS mounted NAS is the direction
> we'll likely take. It appears (from their online docs) that NetApp can
> have individual volumes up to 16 TB. We should be confirming that with
> them in the next day or two.
>
> HTH,
>
> Joe
From | Date | Subject | |
---|---|---|---|
Next Message | Martijn van Oosterhout | 2004-08-31 21:34:01 | Re: Multicolumn Primary Key |
Previous Message | Mike Nolan | 2004-08-31 21:24:00 | Re: Single Row Table? |