PG performance in high volume environment (many INSERTs and lots of aggregation reporting)

From: Phoenix Kiula <phoenix(dot)kiula(at)gmail(dot)com>
To: pgsql-performance(at)postgresql(dot)org
Subject: PG performance in high volume environment (many INSERTs and lots of aggregation reporting)
Date: 2009-01-28 13:33:50
Message-ID: e373d31e0901280533n670c3337x629849181baafd7@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

[Ppsted similar note to PG General but I suppose it's more appropriate
in this list. Apologies for cross-posting.]

Hi. Further to my bafflement with the "count(*)" queries as described
in this thread:

http://archives.postgresql.org/pgsql-general/2009-01/msg00804.php

It seems that whenever this question has come up, Postgresql comes up
very short in terms of "count(*)" functions.

The performance is always slow, because of the planner's need to guess
and such. I don't fully understand how the statistics work (and the
explanation on the PG website is way too geeky) but he columns I work
with already have a stat level of 100. Not helping at all.

We are now considering a web based logging functionality for users of
our website. This means the table could be heavily INSERTed into. We
get about 10 million hits a day, and I'm guessing that we will have to
keep this data around for a while.

My question: with that kind of volume and the underlying aggregation
functions (by product id, dates, possibly IP addresses or at least
countries of origin..) will PG ever be a good choice? Or should I be
looking at some other kind of tools? I wonder if OLAP tools would be
overkill for something that needs to look like a barebones version of
google analytics limited to our site..

Appreciate any thoughts. If possible I would prefer to tone down any
requests for MySQL and such!

Thanks!

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Robert Haas 2009-01-28 14:08:50 Re: LIKE Query performance
Previous Message A. Kretschmer 2009-01-28 09:43:19 Re: LIKE Query performance