From: | dg(at)illustra(dot)com (David Gould) |
---|---|
To: | maillist(at)candle(dot)pha(dot)pa(dot)us (Bruce Momjian) |
Cc: | maillist(at)candle(dot)pha(dot)pa(dot)us, hackers(at)postgreSQL(dot)org |
Subject: | Re: [HACKERS] tables >2GB |
Date: | 1998-03-19 20:09:43 |
Message-ID: | 9803192009.AA07425@hawk.illustra.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
> > Now that we know the storage manager code that splits tables over 2GB
> > into separate files doesn't work(Irix), can we rip out that code and
> > just use the OS code to access >2GB files as normal files. Now, most
> > OS's can support 64-bit files and file sizes.
> >
> > Because it is isolated in the storage manager, it should be easy.
>
> Can someone knowledgeable make a patch for this for our mega-patch?
>
There are still quite a few OS's out there that do not support >2GB files
yet. Even my beloved Linux (x86)...
So, how about we fix the storage manager instead?
A neat thing that Illustra does is allow you to stripe a table across
multiple directories. You get big tables, easy storage management, and
a nice performance boost.
create stripedir('stripe1', '/disk1/data/stripe1');
create stripedir('stripe2', '/disk2/data/stripe2');
create table giant_table (...) with (stripes 4, 'stripe1', 'stripe2');
-- the '4' is the number of pages to interleave.
Then the smgr just distributes the blocks alternately across the stripes.
read_block(blockno, ...stripeinfo)
{
...
stripe = (blockno / stripe_interleave ) % number_of_stripes;
stripe_block = blockno / number_of_stripes;
fd = stripe_info->fd[stripe];
lseek(fd, stripe_block * BLOCKSIZE, SEEK_SET);
...
}
All vastly oversimplified of course....
-dg
David Gould dg(at)illustra(dot)com 510.628.3783 or 510.305.9468
Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612
- I realize now that irony has no place in business communications.
From | Date | Subject | |
---|---|---|---|
Next Message | The Hermit Hacker | 1998-03-19 20:14:24 | Re: [HACKERS] tables >2GB |
Previous Message | Bruce Momjian | 1998-03-19 19:57:13 | Re: AW: [HACKERS] varchar() vs char16 performance |