emphasis on robustness, availability, and error handling under a heavy
load, rather than on what would be considered correct behavior under
normal circumstances. In particular, the goals of such tests may be to
ensure the software doesn't crash in conditions of insufficient computational
resources (such as memory or disk space), unusually high concurrency, or
denial of service attacks.
Stress testing is a type of performance testing focused on determining an
application’s robustness, availability, and reliability under extreme conditions.
The goal of stress testing is to identify application issues that arise or become
apparent only under extreme conditions. These conditions can include heavy
loads, high concurrency, or limited computational resources. Proper stress
testing is useful in finding synchronization and timing bugs, interlock problems,
priority problems, and resource loss bugs. The idea is to stress a system to the
breaking point in order to find bugs that will make that break potentially
harmful. The system is not expected to process the overload without adequate
resources, but to behave (e.g., fail) in an acceptable manner
(e.g., not corrupting or losing data).
Stress tests typically involve simulating one or more key production scenarios
under a variety of stressful conditions. For example, you might deploy your
application on a server that is already running a processor-intensive application;
in this way, your application is immediately “starved” of processor resources
and must compete with the other application for processor cycles. You can also
stress-test a single Web page or even a single item such as a stored procedure
or class.
Examples of Stress Conditions
Examples of stress conditions include:
* Excessive volume in terms of either users or data; examples might include
a denial of service (DoS) attack or a situation where a widely viewed
news item prompts a large number of users to visit a Web site during a
three-minute period.
* Resource reduction such as a disk drive failure.
* Unexpected sequencing.
* Unexpected outages/outage recovery.
Examples of Stress-Related Symptoms
Examples of stress-related symptoms include:
* Data is lost or corrupted.
* Resource utilization remains unacceptably high after the
stress is removed.
* Application components fail to respond.
* Unhandled exceptions are presented to the end user.
Input
To perform stress testing, you are likely to use as reference
one or more of the following items:
* Results from previous stress tests
* Application usage characteristics (scenarios)
* Concerns about those scenarios under extreme conditions
* Workload profile characteristics
* Current peak load capacity (obtained from load testing)
* Hardware and network architecture and data
* Disaster-risk assessment (e.g., likelihood of blackouts, earthquakes, etc.)
Output
Output from a stress test may include:
* Measures of the application under stressful conditions
* Symptoms of the application under stress
* Information the team can use to address robustness, availability,
and reliability
Approach for Stress Testing
The following steps are involved in stress-testing a Web application:
Step1 - Identify test objectives. Identify the objectives of stress
testing in terms of the desired outcomes of the testing activity.
Step2 - Identify key scenario(s). Identify the application scenario or
cases that need to be stress-tested to identify potential problems.
Step3 - Identify the workload. Identify the workload that you want to
apply to the scenarios identified during the “Identify objectives” step.
This is based on the workload and peak load capacity inputs.
Step4 - Identify metrics. Identify the metrics that you want to collect about
the application’s performance. Base these metrics on the potential
problems identified for the scenarios you identified during the
“Identify objectives” step.
Step 5 - Create test cases. Create the test cases in which you define steps
for running a single test, as well as your expected results.
Step 6 - Simulate load. Use test tools to simulate the required load for
each test case and capture the metric data results.
Step 7 - Analyze results. Analyze the metric data captured during the test.
Summary
Stress testing allows you to identify potential application issues that surface only
under extreme conditions. Such conditions range from exhaustion of system
resources such as memory, processor cycles, network bandwidth, and disk
capacity to excessive load due to unpredictable usage patterns, common in
Web applications.
Stress testing centers around objectives and key user scenarios with an
emphasis on the robustness, reliability, and stability of the application.
The effectiveness of stress testing relies on applying the correct methodology
and being able to effectively analyze testing results. Applying the correct
methodology is dependent on the capacity for reproducing workload
conditions for both user load and volume of data, reproducing key scenarios,
and interpreting the key performance metrics.
