Re: [GENERAL] PostgreSQL + IDS load/latency help

From: Dennis Gearon <gearond(at)cvc(dot)net>
To: Loki <loki(at)fatelabs(dot)com>
Cc: focus-ids(at)securityfocus(dot)com, pgsql-general(at)postgresql(dot)org, pgsql-admin(at)postgresql(dot)org
Subject: Re: [GENERAL] PostgreSQL + IDS load/latency help
Date: 2003-03-27 16:49:19
Message-ID: 3E832B8F.9090205@cvc.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin pgsql-general

what do you mean by 'events'?

Loki wrote:
> Lists:
>
> We have a PostgreSQL database full of over 500,000 events. When the
> database has a small number of events, to retrieve a specific event
> stored in that db takes a mere few seconds.
> However, as the database grew in size to where it is now, it takes over
> 15-20 seconds to get information back from a query to that database. We
> have tried everything, including vacuum, which someone else recommended
> to me.
>
> I noticed when running top(exec) on the system that the PostgreSQL
> process eventually eats up ALL available RAM (half a gig of RAM) and
> forces the system into scratch space. When rebooting the machine the
> query time is greatly improved back down to a reasonable query time,
> (obviously because it hasnt eaten up all the memory yet.)
>
> Am I correct in saying that 500,000 events shouldn't be a problem? Is
> there anyone out there with more than 500,000 events in their DB than
> what we currently have?
>
> Can anyone recommend anything to try or do to rectify the situation.
> 15-20 seconds to get the information queried from the database is
> unacceptable and can not work. Please advise.
>
> * We are running Snort 1.9.1 storing events to a custom PostgreSQL
> database. Dumping the database and creating a new one every so often is
> also not an option. Can anyone provide assistance? Please advise.
>
> For those needing more details on our setup, etc. I am not currently
> able to log into the system. Please email me and I can provide you more
> details offline.
>
> Sincerely,
>
> Eric Hines
> Internet Warfare and Intelligence
> Fate Research Labs
> http://www.fatelabs.com
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faqs/FAQ.html
>

In response to

Browse pgsql-admin by date

  From Date Subject
Next Message Hacksaw 2003-03-27 18:17:10 Upgrading: So now you tell me!!?!?
Previous Message Gavin M. Roy 2003-03-27 16:23:57 Re: PostgreSQL + IDS load/latency help

Browse pgsql-general by date

  From Date Subject
Next Message Dennis Gearon 2003-03-27 16:55:26 Re: Solution to UPDATE...INSERT problem
Previous Message Gavin M. Roy 2003-03-27 16:23:57 Re: PostgreSQL + IDS load/latency help