sizing storage for 100 M files

From: Alinga Yeung <alinga(at)uvic(dot)ca>
To: pgsql-novice(at)postgresql(dot)org
Subject: sizing storage for 100 M files
Date: 2009-09-17 21:48:38
Message-ID: 4AB2AEB6.6060000@uvic.ca
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-novice

Hi,

We plan to acquire some storage for our storage system to be based on
iRODS + PostgreSQL. iRODS uses PostgreSQL to store the metadata of a
file. For each file that iRODS stores, a row is added to each of the
following tables in PostgreSQL.

create table R_DATA_MAIN
(
data_id bigint not null,
coll_id bigint not null,
data_name varchar(1000) not null,
data_repl_num INTEGER not null,
data_version varchar(250) DEFAULT '0',
data_type_name varchar(250) not null,
data_size bigint not null,
resc_group_name varchar(250),
resc_name varchar(250) not null,
data_path varchar(2700) not null,
data_owner_name varchar(250) not null,
data_owner_zone varchar(250) not null,
data_is_dirty INTEGER DEFAULT 0,
data_status varchar(250),
data_checksum varchar(1000),
data_expiry_ts varchar(32),
data_map_id bigint DEFAULT 0,
data_mode varchar(32),
r_comment varchar(1000),
create_ts varchar(32),
modify_ts varchar(32)
);

And:

create table R_OBJT_ACCESS
(
object_id bigint not null,
user_id bigint not null,
access_type_id bigint not null,
create_ts varchar(32),
modify_ts varchar(32)
);

Size of R_DATA_MAIN ~= 7618 bytes
Size of R_OBJT_ACCESS ~= 88 bytes

According to the PostgreSQL FAQ, a PostgreSQL database may require up to
five times the disk space to store data from a text file.

Is it valid for me to use this multiplier to estimate the storage
requirement for 100 M files?
i.e. estimated storage = (7618 bytes + 88 bytes) * 100 M * 5
= 3.853 TB

Thank you in advance for your help.

Alinga

Responses

Browse pgsql-novice by date

  From Date Subject
Next Message Stephan Szabo 2009-09-17 22:28:19 Re: sizing storage for 100 M files
Previous Message Alinga Yeung 2009-09-17 21:45:04 sizing storage for 100 M files