big wide table

From: Michael Lush <mjlush(at)gmail(dot)com>
To: pgsql-novice(at)postgresql(dot)org
Subject: big wide table
Date: 2011-12-08 12:55:38
Message-ID: CACXX7MfzzVP9tbnmjs3jzGQttrtA_qxmCghbVFpO-xO9hfWePg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-novice

I have a dataset with ~100,000 columns and ~200,000 rows (GWAS study
data(1))
none of the fields have more than about 15 characters

Sample1,A

I want to be able to extract subsets of columns and rows for analysis

I can see two ways I can approach this:-

Convert it to something RDF like ie

sample_id
column_number
data

to make a table with 3 columns and 20 billion rows

or use the array datatype

sample_id
array_of_data

(1) http://en.wikipedia.org/wiki/Genome-wide_association_study

--
Michael

Browse pgsql-novice by date

  From Date Subject
Next Message Ioannis Anagnostopoulos 2011-12-08 12:57:05 What is faster?
Previous Message Jude Lucien 2011-12-08 11:15:10 Re: Basic Query Question