Skip site navigation (1) Skip section navigation (2)

Re: [PERFORM] Postgres and really huge tables

From: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Brian Hurt <bhurt(at)janestcapital(dot)com>, pgsql-performance(at)postgresql(dot)org, pgsql-advocacy(at)postgresql(dot)org
Subject: Re: [PERFORM] Postgres and really huge tables
Date: 2007-01-19 10:03:05
Message-ID: Pine.LNX.4.64.0701191258070.18339@sn.sai.msu.ru (view raw or flat)
Thread:
Lists: pgsql-advocacypgsql-performance
On Thu, 18 Jan 2007, Tom Lane wrote:

> Brian Hurt <bhurt(at)janestcapital(dot)com> writes:
>> Is there any experience with Postgresql and really huge tables?  I'm
>> talking about terabytes (plural) here in a single table.
>
> The 2MASS sky survey point-source catalog
> http://www.ipac.caltech.edu/2mass/releases/allsky/doc/sec2_2a.html
> is 470 million rows by 60 columns; I don't have it loaded up but
> a very conservative estimate would be a quarter terabyte.  (I've
> got a copy of the data ... 5 double-sided DVDs, gzipped ...)
> I haven't heard from Rae Stiening recently but I know he's been using
> Postgres to whack that data around since about 2001 (PG 7.1 or so,
> which is positively medieval compared to current releases).  So at
> least for static data, it's certainly possible to get useful results.
> What are your processing requirements?

We are working in production with 2MASS and other catalogues, and
2MASS is not the biggest. The nomad catalog has more than milliard records.
You could query them online
http://vo.astronet.ru/cas/conesearch.php
Everything is in PostgreSQL 8.1.5 and at present migrate to the 8.2.1,
which is very slow, since slow COPY.
The hardware we use is HP rx1620, dual Itanium2, MSA 20, currently
4.5 Tb.



>
> 			regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

In response to

pgsql-performance by date

Next:From: Merlin MoncureDate: 2007-01-19 13:45:13
Subject: Re: DB benchmark and pg config file help
Previous:From: Arjen van der MeijdenDate: 2007-01-19 06:15:12
Subject: Re: Configuration Advice

pgsql-advocacy by date

Next:From: Jean-Paul ArgudoDate: 2007-01-19 12:39:04
Subject: Re: PostgreSQL booth, Solutions Linux 2007, Paris
Previous:From: Denis LussierDate: 2007-01-19 05:36:24
Subject: EnterpriseDB Apology

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group