Re: fine tuning for logging server

On Wed, 2005-03-30 at 20:11 -0600, John Arbash Meinel wrote:
> Florin Andrei wrote:
> >
> >For performance reasons, i was thinking to keep the tables append-only,
> >and simply rotate them out every so often (daily?) and delete those
> >tables that are too old. Is that a good idea?
> >
> If you aren't doing updates, then I'm pretty sure the data stays packed
> pretty well. I don't know that you need daily rotations, but you
> certainly could consider some sort of rotation schedule.

(sorry for re-asking, i'm coming from a mysql mindset and i still have a
lot to learn about pgsql)

So, it is indeed a bad idea to delete rows from tables, right? Better
just rotate to preserve the performance.

Daily rotation may simplify the application logic - then i'll know that
each table is one day's worth of data.

> The biggest performance improvement, though, is probably to group
> inserts into transactions.

Yes, i know that. I have little control over the apps, though. I'll see
what i can do.

> How fast is fast? How many log messages are you expecting? 1/s 100/s 1000/s?

More is better. <shrug>
I guess i'll put it together and give it a spin and see just how far it
goes.

I actually have some controls over the data that's being sent (in some
places i can limit the number of events/second), so that might save me
right there.

--
Florin Andrei

http://florin.myip.org/