Software Performance Testing

---Sathireddy Mandadi Satish.mandadi@gmail.com

Possible over-interaction between application servers and database servers Sizing of packets (size in application versus size on network) Connection pools and connection sharing Location of servers on the network Increased latency and jitter Application issues: Excessive memory allocation and de-allocation Improper task initialization and housekeeping Improper garbage collection, especially after a fault or failure Loss of sessions or sessions kept alive too long (timers) Persistent applications like BlackBerry devices Application configuration parameter conflicts High resource consumption features (CAD/CAM, etc.) Database issues: Index design Use of dynamic indexes Potential deadlocks from locking contention Inefficient use of cache Overuse of cache Use of resource-intensive features such as referential integrity Use of stored procedures or trigger events Table fragmentation (overuse of third normal form) Improper timing of table and index reorganization

WHAT IS PERFORMANCE TESTING Performance Testing determines how fast a system

performs under a particular workload to validate the quality attributes of the system such as scalability and reliability and Resource usage of the Application Under Test. It explores several system qualities, that can be simplified to: Speed - does the system respond quickly enough? Capacity - is the infrastructure sized satisfactorily ? Scalability - can the system grow to handle future volumes? Stability - does the system behave correctly under load?

Why Do Performance Testing?

Obtain an accurate picture of end-to-end system performance
before going live. How The System is Capable of handling future growth. Whether Application requires a performance improvement/hardware upgrade prior to release. Revenue losses or damaged brand credibility due to scalability or stability issues. Customers being dissatisfied with application response time. Analyzing the behavior of the application at various load levels. Identifying bottlenecks in the application.

Why do performance testing?

May not detect some functional defects that only

appear under load. Does the application respond quickly enough for the intended users? Will the application handle the expected user load and beyond? Will the application handle the number of transactions required by the business? Is the application stable under expected and unexpected user loads?

Performance Tester Must Know

What is the type of communication between the client and server? Requirements to monitor & Transaction Response Time or server
utilization (SLA). Is the application client-side?, And the AUT architecture. Is there client-side activities involved in the entire application? What is the authentication mechanism?

Does the application allow multiple login of a single user? How does the application maintain a session?. Environment specifications including Software and hardware. How many users or Volumetric client expecting? How does the load balancer distribution the load? What are the parameters been sent to the Server?

Loadrunner Benifits
HP LoadRunner supports performance testing for a wide range of
Decreases the risk of deploying systems that do not meet performance
application environments and protocols
requirements Reduces hardware and software costs by accurately predicting system capacity. Load Runner reduces the personnel requirements b) Load Runner reduces the hardware requirements c) The controlling will be accomplished effectively by the controller d) Because Load Runner tests are fully automated, we can easily repeat them as many times we need

Lets you begin intelligent service-level management before services go live Shortens test cycles to accelerate delivery of high quality applications Pinpoints end-user, system-level and code-level bottlenecks rapidly and
with ease. Reduces the cost of defects by early testing in the development cycle

Loadrunner Versions
LR Version 6.0--- Used a separate thread per user, which required almost 10 times

more i/o and CPU cycles than 6.5. Version 6.5 available June 2000 Turbo Load technology-thousands of vusers using a single operating system thread. Version 7.8 became available September 2003. Version 7.8 Feature Pack 1 added support for Windows XP. Version 8.0 became available August 2004 Additional Runtime Settings & Diagnostics and tuning Capabilities. Version 8.1 became available October 2005 Workflow view and workflow wizard renamed as Execution log and replay log. Version 8.1 Feature Pack 3 Web services Enhancements. Version 8.1 Feature Pack 4Upgraded recorder version. Version 8.1 Feature Pack 4 --January 2, 2007. Version 9.10 installer, available Feb. 2008. Version 9.50 available June 2009. Version 9.51 patch to 9.50 --July 6, 2009, enables better AJAX recognition and specification of DOM element properties. which require Vista users to allow.

Features of Load runner 9.5

Protocol Advisor : Protocol Advisor helps you to determine an appropriate

protocol for recording a Vuser script. The Protocol Advisor scans your application for elements of different protocols and displays a list of the detected protocols. We can export the Test Result from VUGen scripts to HTML and open Quality Center defects directly from the report. Load Runner 9.5 supports Windows Vista SP1. Provides support upto .NET Framework 3.5 version and supports .NET and Web Services protocols. LoadRunner is now integrated with a 3rd party software Shunra, to provide you access to WAN emulation. LR controller has a new utility called Host Security Manager utility that will allow you to create secure communication channels between the LoadRunner Controller and load generators. There is a new Analysis API that will allow you to create an analysis session from test results LR analysis now supports SQL Server 2005. LoadRunner 9.5 no longer supports Windows 2000. LR 9.5 is compatible with Quality Center QC) 9.2, 10.0 and QTP 9.5 and 10.0 versions.

Key Types of Performance Testing

Performance test :
A performance test is a technical investigation done to determine or validate the response, speed, scalability, stability characteristics of the product under test. Focuses on determining if the user of the system will be satisfied with the performance characteristics of the application. Identifies mismatches between performance-related expectations and reality. Supports tuning, capacity planning, and optimization efforts.
Purpose : To determine or validate speed, scalability, and/or stability.

 Load test:
A load test enables you to measure response times, throughput rates, and resource-utilization levels, and to identify your applications breaking point, assuming that the breaking point occurs below the peak load condition. Performance objectives are often specified in a service level agreement (SLA).

Purpose:

To verify application behavior under normal and peak load conditions.
Determines the sufficiency of a hardware environment. Evaluates the adequacy of a load balancer. Detects concurrency issues. Determines how many users can application handle.

Detects functionality errors under load.

 Stress test :
shows how the application behaves under extreme load conditions. Purpose: To determine or validate an applications behavior when it is pushed beyond normal or peak load conditions.

Stress testing enables you to identify your applications weak points, and

Determines the side effects of common hardware or supporting application failures.

Determines if data can be corrupted by overstressing the system. Helps to determine what kinds of failures are most valuable to plan for.
Challenges : Because stress tests are unrealistic by design, some stakeholders may dismiss test results.

It is often difficult to know how much stress is worth applying.

Capacity test:

Capacity testing helps you to identify a scaling strategy in order to

determine whether you should scale up or scale Down. Capacity testing is conducted in conjunction with capacity planning, which you use to plan for future growth, such as an increased user base or increased volume of data. For example, to accommodate future loads, you need to know how many additional resources (such as processor capacity, memory usage, disk capacity, or network bandwidth) are necessary to support future usage levels.

SOAK TEST/Endurance Test

Soak testing involves testing a system with a
significant load extended over a significant period of time, to discover how the system behaves under sustained use. For Example: a system may behave exactly as expected when tested for 1 hour. However, when it is tested for 3 hours, problems such as memory leaks cause the system to fail or behave randomly.

Performance Testing Approach

Performance Test Approach

Performance Testing Tools

Load Runner Rational Robot Silk Performer Web Load OpenSTA Visual Studio Team System JMeter

Comparision of features of Loadrunner, Silk Performer and RPT Features Sl no 1: Protocol Support Load Runner : Supports over 51 protocols Silk Perfomer: Supports around 40 protocols RPT :Very few - HTTP, SAP, Sockek API, Citrix Sl no 2 : Selection of multiprotocol for scripting Load Runner : Supported Silk Perfomer: Not supported RPT : Not supported Sl no 3 : Data corelation Load Runner : Much simpler and easier Silk Perfomer : Clumsy RPT : Simpler Sl no 4 : Fune tuning Load Runner :HP Sitescope and Diagnostics available for troubleshooting the identified bottlenecks Silk Perfomer:Not possible, rely on 3rd party tool RPT :To use IBM's Tivoli Sl no 5 : Identifiation of bottlenecks Load Runner :Easier and faster Silk Perfomer:Not easy RPT :Not easy Sl no 6 : Analysis Load Runner :Around 100 types garphs present. Cross garph result.Help in finding the bottle neck. Silk Perfomer:Supports cross graphs RPT :Has limitation Sl no 7 : Analysis report Load Runner :Repors - html,excel and word Silk Perfomer:Only html RPT :Only html Sl no 8 : Industry-standard scripting language Load Runner : Load Runner:C,Java,VB Silk Perfomer: bdl RPT : Java Sl no 9 : Support level Load Runner : Three levels of support available. Good support provided Silk Perfomer: Not too good in providing support RPT : Not too good in providing support Sl no 10 : Latency Time analysis Load Runner : End to end response time as well as response time at each level supported. Silk Perfomer: Displays response time at end to end as well as in the method level RPT : Displays response time end to end Sl no 11 : Distributed Load Generation Load Runner : Can control multiple load generators and collect results. Can control load generators located at remote networks (through a firewall) if required Silk Perfomer: Single MMC can control upto 256 agents (load injectors) provided sufficient physical memory is available on the MMC. RPT : Not Supported

Understanding LoadRunner Licensing

Load Runner licenses fall into one of the following categories: permanent license-Annual maintenance fee required Term license -either 1 month or 3 months Virtual User Days ex:1000 vusers for one day Software as a Service :- Hp will provide the controller and charge When you purchase a Load Runner license, you will need to purchase a license for a Load Runner Controller and for a certain number of Virtual Users of a particular type.

Performance Testing Process

The following steps are involved in load-testing:

Step 1 - Identify performance acceptance criteria Step 2 - Identify key scenarios Step 3 - Create a workload model Step 4 - Identify the target load levels Step 5 - Identify metrics Step 6 - Design specific tests Step 7 - Run tests Step 8 - Analyze the results

Loadrunner Testing Process

Performance and Technical Testing Approach

Loadrunner Testing Process

Load testing typically consists of five phases: planning, script creation, scenario definition, scenario execution, and results analysis. Define your performance testing requirements, for example, number of concurrent users, typical business processes and required response times. Create Vuser Scripts: Capture the end-user activities into automated scripts. Define a Scenario: Use the LoadRunner Controller to set up the load test environment. Run a Scenario: Drive, manage, and monitor the load test from the LoadRunner Controller. Analyze the Results: Use LoadRunner Analysis to create graphs and reports, and evaluate the performance.

LoadRunner Internal Architecture