Re: Two weeks to feature freeze

Nigel J. Andrews wrote:

>On Thu, 26 Jun 2003, Thomas Swan wrote:
>
>
>>Is it possible the sourceforge compile farms could be used for some of
>>the automated testing? I'm not sure how that system works, but it could
>>be worth looking into.
>>
>>
>
>Isn't the sourceforge license very scary and along the lines of "whatever you
>put on here we own it's just we tend not to persue that at the moment as
>there's not much money in it for us but that doesn't stop us from claiming it
>at some indeterminate time in the future"?
>
If it's that intrusive, then it was a bad idea. But, I didn't find
anything like that on their Terms of Use
<http://sourceforge.net/docman/display_doc.php?docid=6048&group_id=1>
page. The compiler farm has a relatively small number of platforms, but
perhaps it would be enough to get started with at least verifying an
automated test would work. See Guide to the Sourceforge Compile Farm
<http://sourceforge.net/docman/display_doc.php?docid=762&group_id=1>.

In terms of implementation, I was thinking of something like the following.

* clean the source, destination directories
* pull latest CVS tip down.
* record environment / installed packages
* loop - on different options ( w/ or w/o krb5, w/ or w/o ssl, etc. )
o make clean
o configure with sets of options
o compile
+ log messages
+ analyze errors ( perhaps gather statitistics:
warnings, failures, notices, etc.)
o (run / install) if successful
o run tests
+ output results (perhaps to HTML)
+ compare results with expected
+ record differences if any | gather aggregate information
o uninstall / clean up
* end loop

Perhaps there could be an occasion where the test would be able to put
in a corrupt WAL or a corrupt table to do regression tests for recovery
of errors.

Of course, these are just ideas and I'm not sure how practical it is to
do any of them. I just am really concerned about the uninstall/clean up
phase and how that can be done in an orderly fashion. Unless the
process can start from a clean state again, then it won't be valid. At
one point I had even given thought, vainly, to purchasing VMWare for
such an occasion. Suggestions?