Skip site navigation (1) Skip section navigation (2)

Re: Static mirror generation

From: "Dave Page" <dpage(at)vale-housing(dot)co(dot)uk>
To: "Alexey Borzov" <borz_off(at)cs(dot)msu(dot)su>, <pgsql-www(at)postgresql(dot)org>
Subject: Re: Static mirror generation
Date: 2004-06-18 08:04:43
Message-ID: E7F85A1B5FF8D44C8A1AF6885BC9A0E40C360D@ratbert.vale-housing.co.uk (view raw or flat)
Thread:
Lists: pgsql-www
 

> -----Original Message-----
> From: pgsql-www-owner(at)postgresql(dot)org 
> [mailto:pgsql-www-owner(at)postgresql(dot)org] On Behalf Of Alexey Borzov
> Sent: 17 June 2004 15:19
> To: pgsql-www(at)postgresql(dot)org
> Subject: [pgsql-www] Static mirror generation
> 
> Greetings.
> 
> I've commited a script for static mirror generation. Unlike 
> previous such scripts, it is just a generic spider that 
> follows links and does not have any knowledge about the site 
> structure. This immediately helped to fix several problems.
> 
> The mirror of website not including docs is generated in ~5 
> minutes. I don't yet know how long it will take with all the 
> docs, got tired after the first 1,5 hours.

Meep, that's slow. The current build takes just a few minutes when the
server is behaving. Still, the idea of using a crawler is a good one -
at least that way nothing will get forgotten, and presumably it will
create a report of any broken links?

> I've set up a proof-of-concept static mirror @ 
> http://oc.cs.msu.su/pgorg/
> 
> The mirror uses Apache's content negotiation, so that if you 
> have Russian set up as the preferred language in your 
> browser, it'll come up in Russian, else in English.

OK, sounds good. Nice work :-)

Regards, Dave.

Responses

pgsql-www by date

Next:From: Alexey BorzovDate: 2004-06-18 08:30:02
Subject: Re: Static mirror generation
Previous:From: Marc G. FournierDate: 2004-06-18 02:02:33
Subject: Re: New News Entry

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group