Re: Concurrency testing

On Wed, Oct 7, 2009 at 10:41 PM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:
>> I don't really see what's wrong with using Perl modules that are
>> likely to be installed most places and easy to obtain where not, if it
>> makes writing a test framework much easier.  But I also think that we
>> should not get bogged down on exactly which tools to use - it seems to
>> me the first thing is to find someone who is willing to do the work.
>> If someone makes an AWESOME test suite that uses a module which is a
>> little too adventurous, we can probably find a way of adjusting it
>> after the fact so as to remove the dependency (I fancy myself fairly
>> good at this sort of thing, where Perl is concerned).  But if we argue
>> about tools now, we're just going to discourage anyone from taking a
>> stab at it.
>
> OK, but what would we be taking a stab at?  Would it simply be
> something like "make dev-check"  or "make concurrency-check", which
> the build farm would then just invoke, thereby killing two birds with
> one stone?  Or would it have to be something fancier than just that?
> Or am I already too lost in the details?

Well, I don't know. I think the first thing would be to figure out
what sort of tests we want to run. Taking a stab in the dark, I'm
thinking that the point of any sort of concurrency testing must be to
check for race conditions, so presumably it's the sort of thing you'd
want to have an easy way to run over and over again in a loop.
Possibly you would also want to test recovery by killing a backend in
the middle somewhere. It may be too much to put into the core tree at
all; it might be easier to maintain as a separate project. But I
don't have a strong opinion on this. The subject of the thread is
"concurrency testing" so it seems to me we ought to start by figuring
out exactly what that means and how we could test it.

...Robert