Re: Advice on machine specs for growth

Rory Campbell-Lange wrote:
> We are looking to upgrade our current database server infrastructure so
> that it is suitable for the next 3 years or so.
>
> We envisage needing about 800GB of primary database storage in the next
> three years, with 1000 databases in the cluster.

1000 is a lot, but should still be ok.

> We are imagining either splitting the cluster into two and (to have four
> main servers) or increasing the disk capacity and RAM in each server.
> The second seems preferable from a day-to-day management basis, but it
> wouldn't be too difficult to deploy our software upgrades across two
> machines rather than one.

If you can scale horizontally by splitting the load across several
independent database servers, then do so by all means.

This may be more administration work initially, but scaling will
come easy when you need it.

It is much more difficult to scale a single monolithic database server.

> Resources on the main machines seem to be perfectly adequate at present
> but it is difficult to know at what stage queries might start spilling
> to disk. We presently occasionally hit 45% CPU utilisation, load average
> peaking at 4.0 and we occasionally go into swap in a minor way (although
> we can't determine the reason for going into swap). There is close to no
> iowait in normal operation.

Disable memory overcommit and set swappiness to 0 on database servers.

> It also seems a bit incongruous writing about physical machines these
> days, but I can't find pricing on a UK data protection compatible cloud
> provider that beats physical price amortised over three years (including
> rack costs). The ability to more easily "make" machines to help with
> upgrades is attractive, though.

I think physical machines are cool.
The resulting system becomes simpler with fewer dependencies, and
it is much easier to debug performance problems.

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com