Re: Hardware recommendations to scale to silly load

From: matt <matt(at)ymogen(dot)net>
To: Bill Moran <wmoran(at)potentialtech(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: Hardware recommendations to scale to silly load
Date: 2003-08-27 14:37:14
Message-ID: 1061995033.15800.138.camel@gibraltar.mattclark.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-performance

> Don't know how "cheap" they are.
>
> I have an app that does large batch updates. I found that if I dropped
> the indexes, did the updates and recreated the indexes, it was faster
> than doing the updates while the indexes were intact.

Yeah, unfortunately it's not batch work, but real time financial work.
If I drop all the indexes my select performance goes through the floor,
as you'd expect.

> Does noatime make much difference on a PostgreSQL database? I haven't
> tested that yet.

Yup, it does. In fact it should probably be in the standard install
documentation (unless someone has a reason why it shouldn't). Who
*cares* when PG last looked at the tables? If 'nomtime' was available
that would probably be a good thing too.

> Can you split it onto multiple boxes? Some database layouts lend themselves
> to this, others don't. Obviously you can't do joins from one server to
> another, so you may lose more in multiple queries than you gain by having
> multiple servers. It's worth looking into though.

I'm considering that. There are some tables which I might be able to
split out. There amy even be some things I can pull from the DB
altogether (session info in particular, so long as I can reliably send a
given user's requests to the same app server each time, bearing in mind
I can't see the cookies too easily because 50% of the requests are over
SSL)

> I know my answers aren't quite the ones you were looking for, but my
> experience is that many people try to solve poor application design
> by simply throwing bigger hardware at the problem. It appears as though
> you've already done your homework, though.

Well, I *hope* that's the case! The core issue is simply that we have
to deal with an insane load for 1 hour a week, and there's just no
avoiding it.

Maybe I can get Sun/HP/IBM to lend some gear (it's a pretty high-profile
site).

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Mendola Gaetano 2003-08-27 14:58:50 SetVariable
Previous Message Fabian Kreitner 2003-08-27 14:36:14 Improving simple textsearch?

Browse pgsql-performance by date

  From Date Subject
Next Message scott.marlowe 2003-08-27 15:17:42 Re: Hardware recommendations to scale to silly load
Previous Message Fabian Kreitner 2003-08-27 14:36:14 Improving simple textsearch?