Application Performance Testing & Conformance

Generally the steps followed are very simple as in any project

1. Plan & Estimate
2. Develop
3. Execute and watch
4. Give your Verdict

What is going to differ across projects/companies depends on how the software development team works.We flow with the full team's efforts , if it is Agile development or Waterfall , be the development in cycles or in a proper top down approach.The end objective too plays a very important role. Rarely do you find set requirements or cross functional teams defining what to performance test.Functionality testing can be tested defined to reach the functional specs set by business. There is no one team which can tell you exactly what they want , a few requirements which need to be related quantitatively. Terminology both from literal and quantitative terms

• System should behave optimally - "How much time does the system get to respond fully? How much time does the system take to show that it is working on your request?"
• System should not crash - "The server should respond back in a reasonable amount of time "
• System should be fault tolerant - what happens when some nodes fail or slow down Does that baloon into a non responsive system?
• Breaking point of the system - Akin to trying to find the weakest link in the chain fidn the most impacted part of the sytem under load
• Capacity Planning - long term plan to test your application and understand any shortcomings in the future and not necessarily with the current load and infrastructure

When a project starts the two paramount things to know are User load pattern and application architecture (components as well as final deployment infrastructure) . Application architecture can change during the course of the tests but the user load pattern would be the final goal to work towards. So in the planning stage we need to do data profiling.If it is a new system most likely the answer is you can get only a projection at the most and not hard numbers.With existing or migrating applications some degree of accuracy can be achieved.By the time the planning and estimation is done we should have the number of users , user actions and load pattern documented(No of users , load distribution, business processes yada yada). There is no use in doing a test which is not the user pattern.

Development is basically trying to simulate the user actions in script form sometimes the scripts are easy to create. sometimes have to do it the old fashioned way of coding it bottom up.Whichever way we take, as long as we can accommodate the user actions in different patterns thats all we care about.It can be with open source tools or LoadRunner or Rational.Working closely with the Development team is better as you dont want to end up trying to figure out if the application cracked or your scripts did when you are running the tests.

Running the load test without the appropriate monitors is like running a blind test.Wherever possible the monitoring should integrated or done in a way to integrate with result analysis to look at the trends.Worst scenario would be to ask someof the admins to watch during the test see any activity or access to the individual system's console to watch during the test. Finally passing or failing a system depends on the set goals.

This is the general process as would be defined in simple terms, things would change depending the on scenario at hand. But roughly this gives an idea of how the the process works.Another big question which keeps popping up is when should the performance test be done - ideally as early as possible would be the best guess.