Before

After

Cost of

UALITY

Business Success through

Quality Software

Case Study
The Client: Test Assessment was done for one of
the largest retail chain in Australia for its fashionable
apparels and homewares catering to the needs of
all Australians. This retail chain is famous for its
quality of the products and makes it affordable for
all the Australians. There are 283 stores across the
nation and they are of 2 categories Urban and
Country stores to cater to the needs of metro and
regional communities.

Area of Engagement

Test Process & Tools Assessment to setup Web Test center

Objectives:

Understand the current test process & identify areas of improvement

Create Implementation road map for setting up Web Test Center
Prepare a Test Strategy to evolve its current testing practice with Industry
Best Practices around People, Process and Tools, that can be referred across
application portfolio

Technology:

Windows XP, Vista

Mac OS
Browsers: IE 6.0, Firefox on
Windows, Firefox & Safari on Mac OS

JavaScript, ASPX
.NET, Visual Studio 2008 (C#)
Liferay, Elcom CMS,
SQL Server 2008

Applications:

Primary web application featuring products, Catalougues, Promotional

information & customer service information
Fashion web site
Shop from home web application
Gift Cards web application
Point of Sale & Back office Applications

Customer Challenges:

Dedicated test team is not available

Metrics are not defined, collected & analyzed
Web test strategies & test processes are not defined & followed

Assessment Approach:

Administer User Diagnostic survey to understand Organizations vision on testing

Understand the current testing process using HCL enhanced TPI framework
Provide AS-IS report
Collect COQ data and analyze
Carryout root cause analysis
Recommend solution themes
Provide implementation roadmap

Engagement Model:

Offshore / Onshore

Engagement Duration:

6 Weeks

Resource Plan:

1 Week Offshore
3 Weeks Onshore
2 Weeks Offshore
Onshore

2
3
2
1

Assessment Outcome

Prioritized focus on Repeatability & Control, Visibility, Scalability & Efficiency

which were rated 2 in the scale of 1 to 5
AS-IS report
Automation readiness rated as 7.94%
UAT found 15% more defects than SIT
Cost of quality 77% (CMMi level 1)
- Appraisal cost 58%
- Prevention cost 0%
- Internal failure cost 16%
- External failure cost 3%
Gaps, Impact Analysis & Recommendations
Provided Solutions
- Test processes
- Test Strategies
Solution roadmaps
- Resource Plan
- Process Implementation plan

Solution Themes
The proposed recommendations are clubbed into solution themes based on AS IS,
Gaps & Impacts analysis with business values and ease of implementation

CoQ Model
To arrive at the COQ value we plotted the cost data in the COQ model
Total build budget spent for Gift cards was $ 79,050.00.
Cost of Good Quality

Cost of Quality=Cost of Good Quality(Cc)+

Cost of Poor Quality(CNC)

Appraisal Costs

Prevention Costs

Peer Reviews and Code

Walkthrough-??

Training (??)

Testing products and

services-58% of Build
errort ($45,849)

Quality Management &

Process Improvement ??

Prototype - 0%

Tool Evaluation and

automation - None

Defect prevention-None

Cost of Quality

Cost of Good
Quality

Prevention Appraisal
Cost
Cost

Cost of Poor
Quality

Internal
Cost

External
Cost

Internal Failure Costs

External Failure Costs

Analysis (??)

Bug Fixing & Re-testing16% - ($12,648)

Production defects
bug fix and re-test-3%
($2,371.50)

Downtime (??)

Customers dissatisfaction

Loss of business

L1/ L2 Support Cost

Litigation and Penalties

Cost of Poor Quality

COQ Analysis Key Assumptions

10 % of development effort for Unit testing.

Total number of defects identified in System and User

would have been more than planned.

Hourly rate of 75 AUD for defect -fix and re-test

5 % defect leakage in Production based on the defect removal efficiency of

System and User testing

testing, effort spent

CoQ Recommendations
To arrive at the current spend on Cost of Quality for Gift card application, we
have analyzed the effort and cost data, Order of Magnitude, SOW, defects,
testplans and test cases.
l

58% of total build budget it

spent on testing

The COST OF QUALITY based on

the above data is around 77%

16% of total build budget

is spent on rework and rest
effort for System and UAT
defects

The COST Spend for building

the product is only 23%

This COQ data maps to a CMMI

level 1Organization as per
Krazner report

3% for fixing and re-testing

production defects

Initiate COQ Measurements

Track and monitor effort spent on testing

Look for opportunities to automate tests

Establish Vendor deliverable review process to ensure product quality

Track production defects and perform root cause analysis

Establish early defect techniques

Solutions Delivered
Test Strategies
defined for

Test Process
defined for

Functional Testing
l Regression Testing
l
Automation Testing
l
Performance Testing
l
Usability Testing
l
Security Testing
l
Failover Testing
l
Additional Web
testing
- Link Test
- Cache Test
- Content Verification
- Complex Testing
- Cookie Testing

Test Life Cycle

- Enhanced V Model
- Tailored test
process
l Estimation Process
l Test case design &
Risk based testing
techniques
l Test governance
l Reporting
l Allied Processes
- Configuration
Mgmt.
- Change Mgmt.
- Release Mgmt.
l Defect Management
l Quality & Knowledge
Mgmt.
l Review Processes
l Standards,
Templates &
Checklists

Test Automation Cost of Quality

- Initiative
Implementation
Proof of Concept
(POC)
l Pilot Automation
l Automation
Deployment &
Maintenance
l ROI tracking
l

Metrics Definition
l Test Metrics
- Quality
- Delivery
- Business
- Productivity

CoQ Defect
Analysis
l Indicative CoQ
Model
l Recommendations
l

Solution
Roadmaps
l Resource Plan
l Process
Implementation

Hello, Im from HCL!. We work behind the scenes, helping our customers to shift paradigms and start revolutions. We use
digital engineering to build superhuman capabilities. We make sure that the rate of progress far exceeds the price. And
right now, 64,000 of us bright sparks are busy developing solutions for 500 customers in 26 countries across the world.
How can I help you?
www.hcltech.com