Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] tables >2GB

From: The Hermit Hacker <scrappy(at)hub(dot)org>
To: David Gould <dg(at)illustra(dot)com>
Cc: Bruce Momjian <maillist(at)candle(dot)pha(dot)pa(dot)us>, hackers(at)postgreSQL(dot)org
Subject: Re: [HACKERS] tables >2GB
Date: 1998-03-19 20:14:24
Message-ID: Pine.NEB.3.95.980319151404.19757C-100000@hub.org (view raw or flat)
Thread:
Lists: pgsql-hackers
Does this act as a notice of volunteering to work on this aspect of the
code? *Grin*


On Thu, 19 Mar 1998, David Gould wrote:

> > > Now that we know the storage manager code that splits tables over 2GB
> > > into separate files doesn't work(Irix), can we rip out that code and
> > > just use the OS code to access >2GB files as normal files.  Now, most
> > > OS's can support 64-bit files and file sizes.
> > > 
> > > Because it is isolated in the storage manager, it should be easy.
> > 
> > Can someone knowledgeable make a patch for this for our mega-patch?
> > 
> 
> There are still quite a few OS's out there that do not support >2GB files
> yet. Even my beloved Linux (x86)...
> 
> So, how about we fix the storage manager instead?
> 
> A neat thing that Illustra does is allow you to stripe a table across
> multiple directories. You get big tables, easy storage management, and
> a nice performance boost.
> 
> create stripedir('stripe1', '/disk1/data/stripe1');
> create stripedir('stripe2', '/disk2/data/stripe2');
> 
> create table giant_table (...) with (stripes 4, 'stripe1', 'stripe2');
>  -- the '4' is the number of pages to interleave.
> 
> Then the smgr just distributes the blocks alternately across the stripes.
> 
> read_block(blockno, ...stripeinfo)
> {
>    ...
>    stripe = (blockno / stripe_interleave ) % number_of_stripes;
>    stripe_block = blockno / number_of_stripes;
> 
>    fd = stripe_info->fd[stripe];
>    lseek(fd, stripe_block * BLOCKSIZE, SEEK_SET);
>    ...
> }
> 
> All vastly oversimplified of course....
> 
> -dg
> 
> David Gould            dg(at)illustra(dot)com           510.628.3783 or 510.305.9468 
> Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
>  - I realize now that irony has no place in business communications.
> 
>    
>    
> 


In response to

pgsql-hackers by date

Next:From: Michael J SchoutDate: 1998-03-19 20:50:08
Subject: Bug in DBD::Pg v0.69?
Previous:From: David GouldDate: 1998-03-19 20:09:43
Subject: Re: [HACKERS] tables >2GB

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group