Re: load / stress testing

Chris:

There are a number of solutions on the market. A company called Soasta
has a cloud based testing solution, my company has a product/solution
that's highly customizable called StressWalk. On another computer I
have a larger list with some ranking info I can send if you are interested.

Having been involved in a number of stress tests for customers, I think
the most important questions in picking a service to help you get this
done are:

* How much value am I likely to get by doing the testing? (If the
app fails in production because your team missed something
fundamental and it takes a day to fix via EIP, how much will that
cost your company in lost revenue and intangibles like reputation?)
* How much money can I afford to spend?
* Do I have good instrumentation in place for my application,
database, infrastructure, etc that I can easily correlate with the
graphs produced by the stress testing application.
* Am I looking for a rubber stamp or real analysis of what's going
on -- do I or my team have the time, expertise and experience in
application, database and infrastructure performance optimization
to do the analysis ourselves?
* Is my application UI difficult to stress test? (ie. Active X
controls, Flex, Java Applet, SilverLight, Cross-site AJAX)
* To create a realistic load profile, do I need to simulate one user
scenario or several?
* Does my application have user session and permission caching
code? If yes, you may need to utilize a solution that can use
large number of user credentials with different permission sets.
If you do not have the users created already and it's a large
enough population, you may want to use the stress testing
service/tool to create and configure the users for your test in
an automated way.

This should get you started. However, if you are leaning toward a
service -- my analysis is that there are a few services priced under
$1000 but they do not look very useful. The services we have found that
look like they can provide some good value to their customers are in the
$2500 - $15,000 range. For really complex stuff like application
components interacting with mainframes and synchronized data extract and
load to facilitate the work, you can expect things from the top end of
that range up to $25-30K.

I'd be happy to have a conversation if you are serious about solving
this problem.

-Jerry

Jerry Champlin
Absolute Performance Inc.
O: (303) 443-7000 x501
C: (303) 588-2547
jchamplin(at)absolute-performance(dot)com

Chris wrote:
> Hi,
>
> Everyone says "load test using your app" - out of interest how does
> everyone do that at the database level?
>
> I've tried playr (https://area51.myyearbook.com/trac.cgi/wiki/Playr)
> but haven't been able to get it working properly. I'm not sure what
> other tools are available.
>
> TIA.