Re: Large databases, performance

On 3 Oct 2002 at 19:33, Shridhar Daithankar wrote:

> On 3 Oct 2002 at 13:56, Nigel J. Andrews wrote:
> > It's one hell of a DB you're building. I'm sure I'm not the only one interested
> > so to satisfy those of us who are nosey: can you say what the application is?
> >
> > I'm sure we'll all understand if it's not possible for you mention such
> > information.
>
> Well, I can't tell everything but somethings I can..
>
> 1) This is a system that does not have online capability yet. This is an
> attempt to provide one.
>
> 2) The goal is to avoid costs like licensing oracle. I am sure this would make
> a great example for OSDB advocacy, which ever database wins..
>
> 3) The database size estimates, I put earlier i.e. 9 billion tuples/900GB data
> size, are in a fixed window. The data is generated from some real time systems.
> You can imagine the rate.

Read that fixed time window..

>
> 4) Further more there are timing restrictions attached to it. 5K inserts/sec.
> 4800 queries per hour with response time of 10 sec. each. It's this aspect that
> has forced us for partitioning..
>
> And contrary to my earlier information, this is going to be a live system
> rather than a back up one.. A better win to postgresql.. I hope it makes it.
>
> And BTW, all these results were on reiserfs. We didn't found much of difference
> in write performance between them. So we stick to reiserfs. And of course we
> got the latest hot shot Mandrake9 with 2.4.19-16 which really made difference
> over RHL7.2..

Well, we were comparing ext3 v/s reiserfs. I don't remember the journalling
mode of ext3 but we did a 10 GB write test. Besides converting the RAID to RAID-
0 from RAID-5 might have something to do about it.

There was a discussion on hackers some time back as in which file system is
better. I hope this might have an addition over it..

Bye
Shridhar

--
"What terrible way to die." "There are no good ways." -- Sulu and Kirk, "That
Which Survives", stardate unknown