Skip site navigation (1) Skip section navigation (2)

sizing storage for 100 M files

From: Alinga Yeung <alinga(dot)yeung(at)nrc-cnrc(dot)gc(dot)ca>
To: pgsql-novice(at)postgresql(dot)org
Subject: sizing storage for 100 M files
Date: 2009-09-17 21:45:04
Message-ID: 4AB2ADE0.6080801@nrc-cnrc.gc.ca (view raw or flat)
Thread:
Lists: pgsql-novice
Hi,

We plan to acquire some storage for our storage system to be based on 
iRODS + PostgreSQL. iRODS uses PostgreSQL to store the metadata of a 
file. For each file that iRODS stores, a row is added to each of the 
following tables in PostgreSQL.

create table R_DATA_MAIN
 (
   data_id bigint not null,
   coll_id bigint not null,
   data_name varchar(1000) not null,
   data_repl_num INTEGER not null,
   data_version varchar(250) DEFAULT '0',
   data_type_name varchar(250) not null,
   data_size bigint not null,
   resc_group_name varchar(250),
   resc_name varchar(250) not null,
   data_path varchar(2700) not null,
   data_owner_name varchar(250) not null,
   data_owner_zone varchar(250) not null,
   data_is_dirty INTEGER DEFAULT 0,
   data_status varchar(250),
   data_checksum varchar(1000),
   data_expiry_ts varchar(32),
   data_map_id bigint DEFAULT 0,
   data_mode varchar(32),
   r_comment varchar(1000),
   create_ts varchar(32),
   modify_ts varchar(32)
 );

And:

create table R_OBJT_ACCESS
 (
   object_id bigint not null,
   user_id bigint not null,
   access_type_id bigint not null,
   create_ts varchar(32),
   modify_ts varchar(32)
 );

Size of R_DATA_MAIN ~= 7618 bytes
Size of R_OBJT_ACCESS ~= 88 bytes

According to the PostgreSQL FAQ, a PostgreSQL database may require up to five times the disk space to store data from a text file.

Is it valid for me to use this multiplier to estimate the storage requirement for 100 M files?
i.e. estimated storage = (7618 bytes + 88 bytes) * 100 M * 5
                       = 3.853 TB

Thank you in advance for your help.

Alinga



pgsql-novice by date

Next:From: Alinga YeungDate: 2009-09-17 21:48:38
Subject: sizing storage for 100 M files
Previous:From: Jignesh ShahDate: 2009-09-17 11:51:23
Subject: How to find the primary/composite keys of table?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group