Re: Postgresql.org search engine.

On Fri, 30 Jan 2004, Josh Berkus wrote:

> Guys,
>
> > Do you have software to do this, including all the inter-posting
> > references and followups? Or do you propose we write this all from
> > scratch?
>
> Robert Bernier apparently wrote something to break up mail for inclusion in a
> database, and should be able to help in a couple months. Josh Drake is also
> willing to help, and has already done a prototype wiithout header searching.

Dumping mail into a database isn't that hard to do ... there are several
projects on the 'Net right now doing that, including one that connects a
POP3 daemon into the database to download the mail ... in fact, from what
I recall of fts.postgresql.org, isn't that what Oleg/Teodor's stuff does?

I'm kinda curious here ... exactly what problem are we trying to solve
here?

Me, I'm just trying to clean up the archives so that when someone gets
their search results, they don't all show the same 'text', which I've
already accomplished ... Dave is working on improving the speed of the
searches, which he has accomplished with ASPseek ...

If I can figure out how to get the Date: of the posting into the
Last-Modified field (I know *how* it should work, but last time I tried it
ended up generating a whack of errors), then that should satisfy Oleg's
beef ...

Oleg, one question ... what do you recommend setting max-age to for
Cache-control? Right now, I have it set to 30 days ... too long? not
long enough?

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy(at)hub(dot)org Yahoo!: yscrappy ICQ: 7615664