Académique Documents
Professionnel Documents
Culture Documents
IBM Software
Agenda
What & why of Regression testing Leveraging Automation
IBM Software
Mobile banking will be the most widely used banking channel by 2020, if not sooner
Will it perform?
The average 2010 automobile contains more lines of software code than a fighter jet
Are we compliant?
Is it secure?
Source: Mobile banking: A catalyst for improving bank performance, Deloitte, 2010; and This Car Runs on Code, Robert Charette, IEEE Spectrum, 2009. 3
IBM Software
Projects cancelled due to poor quality are 15% more costly than successful projects of the same size and type.
Source: Capers Jones, 2011 Based on 675 companies, 35 government/military groups, 13,500 projects, 50-75 new projects/month, 24 countries, 15 lawsuits
4
IBM Software
Regression Testing
The objective of a test is to find defects
New features or software are typically buggy. We want to shake that out
IBM Software
One way to not find defects is to not run any regression tests
IBM Software
IBM Software
Mature products tend to have a lot of expectations about things that have worked in the past
And a lot of tests to prove it
If you run everything thats ever been run in the past (automated or manual)
How long will that take?
How much value does it add? What level of confidence does that imply?
IBM Software
GUI
Stability: Work with the product development team to consider GUI as a feature that needs to be stabilized in iterations
IBM Software
Automation choices
Design your automation solution to leverage your architecture and reduce maintenance
GUI
10
IBM Software
Automating APIs
Test across all technologies, from web services, to mainframe, mobile, etc.
Integration testing
11
11
IBM Software
Automating GUIs
Increase repeatability through automated test playback Test more critical functions faster with automation
Track and communicate progress and regressions throughout the testing lifecycle
IBM Software
Public Cloud
Private Cloud
EJB Business Partners Shared Services
App Under Test Routing Content Service Collaboration Providers Third-party Services Archives Portals File systems
Messaging Services
Directory Identity
Data Warehouse
Mainframe
Heterogeneous Environments
13
IBM Software
Databases
Mainframe applications
Third-party Services
Mainframe
Virtual Services
Heterogeneous Environments
Virtual Services simulate the behavior of an entire application or system during testing
Virtual Services can run on commodity hardware, private cloud, public cloud Each developer, tester can easily have their own test environment Developer and testers continue to use their testing tools (Manual, Web performance, UI test automation)
IBM Software
IBM Software
16
IBM Software
Successfully building tests leads to, um, too many tests Even if its all automated it can take too long to run everything And do you need to ? How to define the right amount of test coverage for a regression suite?
17
IBM Software
Risk assessments captured in Test Plan and Test Cases Base project planning on qualitative risk analysis
Test Case will contain a risk failure score and a risk priority score Documented risk related decisions
IBM Software
IBM Software
Low contribution
Test Suites
2.
3.
20
IBM Software
Doers
Create regression tests Maintain regression tests suites
Automated Doers
Execute regression tests Automate test execution
21
21
IBM Software
Leverage traceability
Story Story
Requirements Requirements
Defects Defects
IBM Software
Doers
Create regression tests update regression suite after each release Maintain regression tests suites re-run the suite and update after each release
Automated Doers
Execute regression tests use tools to integrate regression runs into your builds
23
23
IBM Software
Your team will be honored for a high degree of business value. High software quality will become a product differentiator
$$$
Growth Projects will exceed expectations!
24
IBM Software
Summary
A Goldilocks problem
Its easy to have too much or too little regression testing Prepare for pruning the regression suites
25
IBM Software
26
IBM Software
www.ibm.com/software/rational
Copyright IBM Corporation 2011. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBMs sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
2727