Skip site navigation (1) Skip section navigation (2)

Re: Archives too slow

From: "Greg Sabino Mullane" <greg(at)turnstep(dot)com>
To: pgsql-www(at)postgresql(dot)org
Subject: Re: Archives too slow
Date: 2004-08-29 11:41:14
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-www
Hash: SHA1
> I'm going to look at mirroring it onto the same server that runs
> ... archives is the worst site that we run, since its
> all a bunch of little flat files, so when it gets indexed by the various
> search engines, disk I/O goes through the roof ... we had googlebot index
> it once where we had to literally shut down the server for a few minutes
> while we waited for load to drop ...
On googlebot's page[1], they claim they never go more than once every few
seconds. Surely this should not be a problem as long as these are static
pages. They also have an email address on that page where you can request
that Google go a little gentler on your site.
Also, if the pages are static (or static plus simple cgis), have you considered
using boa? [2] I use it for a large site that has a lot of static pages
and it does great - it's a small, clean, minimal web server written in C.
A final option is an accelerator cache [3]. Not sure if PG is using one
yet, but it probably should be.
- --
Greg Sabino Mullane greg(at)turnstep(dot)com
PGP Key: 0x14964AC8 200408290732

In response to


pgsql-www by date

Next:From: Justin CliftDate: 2004-08-29 12:41:18
Subject: Re: Techdocs site down (db error)
Previous:From: Marc G. FournierDate: 2004-08-29 05:18:29
Subject: Re: Techdocs site down (db error)

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group