Re: Postgres forums ... take 2

Some more changes:

* Added a profile field so you can enter your real name. Your real name is public but optional. If your real name is present then emails sent to the mailing list will use that instead of your forum name.

* Mail parsing script will automatically pick up your real name. This means that posts from the forum are totally transparent with the mailing list.

* Written the script to take a forum post or reply and send it back to the mailing list (using a dummy address for now.) It will change quotation format, remove bbcode, spoof the message-id and in-reply-to fields and look exactly like it's come from an email sender rather than the forum. True interoperability.

* SMTP for outgoing mail is now setup. This should stop any messages from the forums going into your spam box.

* This is the current live mapping of forums <=> mailing lists;
http://postgresql.com.au/ml/info.php
Someone please check these.

* Rewrote the static forum mappers to now use the more robust database mappings which can be updated safely and easily.

* Pending mail tables now use compression (MySQL doesn't support compression so had to implement it myself.)

---

Testers:
Clearly I'm not going to point it to the real mailing list until a bunch of people have checked it and we're all satisfied it's doing exactly as it should. If you want to be a tester these are the steps.
1. Tell me the address you'd like to receive test emails to.
2. Either wait for a topic to be posted or post one yourself (this will then send all the testers an email looking like it's come from a mailing list.)
3. Add some text with the email reply, when your reply is processes it will appear on the forum. [1] Make sure your response matched the post on the forum.
4. Do the reverse, reply on the forums and make sure your emails are formatted correctly. [2]

The real mailing list addresses are not anywhere near these emails so there is no risk sending anything to the real mailing list until I tell it to do so.
Expect some formatting bugs, that's what I want to get sorted out.

[1] I still have the mail parser on manual so if your posts haven't shown up it's not because it's broken, i'm probably asleep.
[2] Forums will probably get moved around, reposted, or deleted. This is just me trying different ways to break it (find bugs.) But I will only mess with threads I've created unless otherwise announcing it.

---

As for the archives...
- There is just under 1.5gb of uncompressed mailing list archives (thats all the pgsql-* mailing lists.)
- Full text indexing doubles that + 1.5gb (quick guess.)
- The compression pending table will probably run around 1gb or less.

Thats a total of about 4gb. Not that I'm saying I would but in case anyone was wondering. I would at least like to load in Jan 2010 onwards as this is still very useful and relevant information to load the search engine with.

---

I think some people misunderstood how the split forums work. All forums are still 100% plugged into the mailing lists, it's more just a categorising thing on the forum software, it makes no difference to the mailing list people as long as the mappings are correct in: http://postgresql.com.au/ml/info.php
This will make more sense if your able to help test.

On 17/11/2010, at 8:15 PM, Magnus Hagander wrote:

> On Wed, Nov 17, 2010 at 00:00, Elliot Chance <elliotchance(at)gmail(dot)com> wrote:
>>
>> On 17/11/2010, at 6:22 AM, Stephen Cook wrote:
>>
>>> On 11/16/2010 10:51 AM, Magnus Hagander wrote:
>>>> What I'm more interested in is still a word from the people who would
>>>> actually *use* a forum on how this would be better than sites like
>>>> Nabble and Gmane.
>>>
>>> I'm one of those. I'm subscribed to these mailing lists simply because it is the only way I know of to get the messages in a timely fashion, but I would greatly prefer a forum-style interface.
>>>
>>> I had never heard of Nabble or Gmane until now, but I just checked them out and from my quick look it *looks* like a web interface for people who prefer mailing lists.
>>>
>>> I like having a category breakdown (at the moment I have my email client splitting the various lists into folders), and I like having little icons telling me which ones I already read and which are new (my email client has that also of course).
>>>
>>> So basically, the email lists are usable, but if this forum works out I'll dump my email subscription in a second and use that. I don't think either is inherently better than the other, it's just personal preference.
>>
>> I'm not sure if anyone is noticing, or just doesn't want to but all that's becoming of the forum is a viewer for the mailing list with the ability to reply. There are already enough forum sites where they shove anything related to postgres into a single generic forum - I see no reason in recreating that.
>
> I think there's a general preference of not fragmenting the discussion
> forums, whether they're in mailinglist of web forum format.
>
> It should certainly not be a single forum for everything. But there
> should be consistent splits.
>
>
>> It's a difficult balancing act to leverage the mailing list community but also use all the features that have made forum software popular in the first place. There will be people who will continue to use mailing list no matter how the forum is presented or functions simply because that's their preferred method, and some people who are used to the different methods of a forum. Everyones input is important, but for the former who are never going to use the forum anyway should have little influence on how it works as forum software.
>
> They should have a *lot* of influence on how the communication between
> the web forum and the mailinglists work. They shouldn't have any
> influence on how the actual forum software works.
>
> But I think you're missing one of the main points - the forums will
> have a significantly reduced value if they don't get responses from
> the people who are currently on the mailinglists. We've had
> disconnected forums before, and they've all died because people have
> posted questions there, and never gotten answers. The part that "the
> mailinglist people" here consider is that this is *worse* for our
> "reputation" than not having the forums at all - having forums that
> don't get responses.
>
>> OK, so solutions? Here in Sydney it's a bit after 9am so I've had time to sleep on it and heres what I'm thinking;
>> - Tagging system. A thread created "Performance of C vs Perl" could be tagged (by a registered user or automated system) as [Performance] [C] [Perl] this would have no impact on the mailing list but make forum viewing and searching more reliable, so a search might be like:
>> Search: "benchmark"
>> Tags: [Perl] [PHP]
>
> A search system can certainly work that way. As long as there's a
> deterministic way of figuring out which mailinglist replies to back
> into, and which threads replies-to-those-replies go.
>
>> For someone looking to find a higher performance solution or comparison between Perl and PHP. I'd rather not do this though because it will require me to change a lot of code in the phpBB3 codebase and still doesn't use a forum in the way its supposed to be used.
>>
>> The way I see it theres no reason why the forums can't be split the way they are now. It makes no difference to the people who will continue to use the mailing list but makes all the difference to forum users who are choosing this forum over others because it has all the backing of the masters on the mailing list in a much better layout of forums than any other site offers.
>
> It may be confusing to the end user, but I'm willing to accept that
> web forum users are used to that :-) As long as there *is* a mapping,
> and that it's consistent, of course.
>
>> There is no perfect solution here, you can't please all the masses all the time. But I do believe there is a workable solution somewhere in the middle.
>
> That is a very good point :-)
>
> --
> Magnus Hagander
> Me: http://www.hagander.net/
> Work: http://www.redpill-linpro.com/