Cheetah Broadband Email Client Test Plan

Version 1.0

Created January 12, 2001

Last Update 08/15/2001

William Gunkel QA Architect

bgunkel@excitehome.net 650-556-3990

Excite@Home 450 Broadway Blvd. Redwood City, CA 94063

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Table of Contents
INTRODUCTION.........................................................................................................................................................3 ASSUMPTIONS.............................................................................................................................................................4 RISKS ...............................................................................................................................................................................5 STRATEGY ....................................................................................................................................................................6 TEST LEVELS ...............................................................................................................................................................7 TEST ENVIRONMENTS ...........................................................................................................................................8 BUGS, PRIORITY & GATE STANDARDS .......................................................................................................11

William Gunkel

Page 2 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Introduction
Overview

This document details the testing responsibilities for the Excite@Home Opus project. This Test Plan is a living document, which can be modified on an as-needed basis. Objectives

Test Plan will be based on Engineering Specification for Project Cheetah, Product Specification Broadband Email Client-aka Cheetah, UI Specifications Excite Broadband Email (Draft Version 0.3 dated Nov 8, 2000), and Marketing Requirements Document Broadband Email (version 1.0) Test cases will be automated as much as possible to assure ease and quickness of testing. Testing of product will be performed during all phase of the release cycle: Alpha, Beta and Final. Test Procedures will be written to document steps for executing the tests Background

Opus is Excite@Homes next generation mail experience (aka Broadband Email), ultimately to be rolled out to and shared in common by Subscribers and open Web Excite users. Project Cheetah: Version 1.0 of our email client software. Cheetah will exist to archive message data on the user desktop and provide an easy to use, responsive user experience that Excite controls (instead of Microsoft or AOL-Netscape). This test plan covers only Cheetah component(s) of Opus project Project Nautilus: Web-based email application designed as a companion to Cheetah. Nautilus will exist to provide download-free access to email from any computer with an Internet connection. Not covered by this test plan.

Scope

This document details the different levels of testing, including functional, boundary, and integration testing. It also details the platforms to be tested against, as well as confi gurations within those platforms. Furthermore, it details specific functionality to be tested within each of the Cheetah components.

William Gunkel

Page 3 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Assumptions
Excite@Home QA is responsible for testing all components of the Cheetah project. Because Cheetah is based on existing email client, which is already stress tested, no stress tests will be applied to the Cheetah email client. Also, Client QA has no control over functionality of server side.

William Gunkel

Page 4 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Risks
Writing the test plan is dependent on communication with Engineering regarding product functionality. Any changes or new features may delay project release date. QA will not be able to create a complete set of test cases, until the first version of Engineering specifications has been completed. HIGH RISK ITEM: Delay in staffing the QA team will have negative effect on the project completion date. Since Cheetah will be available for download and it updates through Regional Data Centers (RDC, or its equivalent), QA will need to simulate the server side functions in a development environment, which mimics the live service. HIGH RISK ITEM: Problems setting up and maintaining test server that mimics the live Email service, will also negatively effect the completion date of the project.

HIGH RISK ITEM: Any problems finding the correct test hardware and configuring to run off the test server (LAN problems, problems OS version, bad hardware etc.) Will negatively effect the completion date of the project.

Any changes, new features, or fixes made after release of the first functional version of Cheetah, may require changes to be made to this test plan thereby having a negative effect on the project completion date.

William Gunkel

Page 5 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Strategy
Excite@Home will write a test plan and develop test cases for @Home project Cheetah. QA will begin to test the iterations once the acceptance criteria is passed. Acceptance criteria project Cheetah must be fully featured (version specific features) and functionality complete before entering QA. Exit Criteria will be met once QA has completed a full suite of tests of the integrated product and all fixed bugs have been regressed. The development team must complete their due diligence by unit testing each product component to verify that the basic features of the product and their respective functions are present and working properly prior to submission to QA for testing. The product code must be "frozen" during testing cycles. During this time, no code changes can be made or the testing cycle will be invalidated. If the product/engineering group violates the code freeze, the testing cycle will be suspended until the code is demonstrated to be frozen.

William Gunkel

Page 6 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Test Levels
Four levels of testing will be applied to each component of the E@H Opus project: Installation, Functional, Boundary, and Integration. Test Level Installation Functional Boundary Integration Scope
Verify that the product installs correctly on all platforms and configurations. Verify that the product functions as specified on all platforms and in all configurations. Verify that the product behaves in a well-defined manner when functional boundaries are approached or crossed Verify that the product behaves as specified when it is used in conjunction with other @Home delivered software.

Features
Auto Configure Auto Update Components: Buddy list, address book Window(s) Sizing Support.com healing agent Sign-in / Sign-out Preferences Drag & Drop Keyboard Navigation Mouse Navigation via buttons/Tool bars Mouse Navigation via menus Message Aging Message deleting / Trash Message Address Message Printing Message sorting Message Copy/Move to/from rules HTML message Formatting & word wrap Message Attachments Message drafts New message polling Notification Spam filtering List scroll bars content List scroll bars resizing List scroll bars screen resolution Resizing columns Personal folders create/delete/move Opening Multi-Personal folders Multi-instances of windows Service not available Standard Windows UI functionality UI functionality & Easy of use UI Message (clarity, spelling etc) Integrated help system Mouse pointers Switching between user accounts Overall UI performance Branding XML resources Uninstall

Functional
X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Boundary
X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Integration
X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

William Gunkel

Page 7 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Test Environments
Operate Systems to be tested Windows95 o Retail/OEM, SR 2.5 Windows98 o Retail/OEM o Second Edition (Retail/OEM) Windows2000 o Retail/OEM, SP 1 Windows ME o Retail/OEM Windows NT Workstation 4.0 o SP 3, SP 4, SP 5, SP 6A

Release vs. Version number

Windows Windows Windows Windows Windows Windows Windows Windows Windows Windows Release 95 OEM 95 SP1 95 Service Release 2 95 Service Release 2.1 95 Service Release 2.5 98 OEM 98 Second Edition Millennium 2000 2000 SP 1 Version 4.00.950 4.00.950A 4.00.1111* (4.00.950B) 4.03.1212-1214* (4.00.950B) 4.03.1214* (4.00.950C) 4.10.1998 4.10.2222A 4.90.3000 5.00.2195 5.00.2195

Browser to be tested IE 5.01 o SP 1 IE 5.5 o

SP 1

William Gunkel

Page 8 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Hardware configurations to be tested Pentium CPU P1 133 MHz P1 133 MHz PII 300 MHz PIII 550 MHz PIII 600 MHz PIII 550 MHz RAM 32 MB 32 MB 64 MB 128 MB 64 MB 128 MB DISK 1.2 GB 150 MB free space 1.2 GB 10 GB 10 GB 10 GB 10 GB Video 800x600 256 colors 800x600 True Color (16 bit) 1024X768 256 colors 1024X768 True Color (16 bit) 1024X768 True Color (24 bit) 1280x1024 True Color (32 bit)

Configuration A Configuration B Configuration C Configuration D Configuration E Configuration F

Notes: Configuration A: Hardware must be found to meet minimum system requirements, as denoted by the Configuration C, D, and E is assume to represent configuration more commonly in use and therefore may vary from the suggested configuration. Configuration F, represents a high end systems and may also vary from the suggested configuration. The hardware used should also be from mixed manufactures. Disk size selection: free space not total disk space is the main parameter for testing. The next standard size may be used.

William Gunkel

Page 9 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Hardware configurations vs. OS (latest releases)

Windows 95 SR 2.5 Windows 98 Second Edition Windows ME Windows 2000 SP 1 NT 4.0 SP 6A

Configuration A Configuration B Configuration C Configuration D Configuration E Configuration F

Full Sub Sub Sub Full Not tested

Full Sub Full Sub Full Sub

Not tested Sub Sub Sub Full Sub

Not tested Sub Sub Sub Full Sub

Not tested Not tested Sub Sub Full Sub

Hardware configuration vs. OS (early releases) Win 95 Configuration A Confi guration E Win 98 Configuration A Configuration E Retail/OEM Sub Sub SR 2.5 Sub Sub

Retail/OEM Sub Sub

Second Edition Sub Sub SP 1 Sub SP 6A Sub

Win 2000 Configuration E Win NT 4.0 Configuration E SP 3 Sub

Retail/OEM Sub SP 4 Sub

SP 5 Sub

A Full suite of test will be run on the configurations as noted. A Sub set of test will be run on configurations as noted. Internet Explorer 5.5 SP1 is used for the full suite of tests. Internet Explorer 5.10 SP 1 and 5.5 SP 1 will be used for the sub-set of tests.

William Gunkel

Page 10 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Bugs, Priority & Gate Standards

For the application under test to be CERTIFIED as an Alpha candidate by Quality Engineering:
o

The application must be installable (this may require making changes to the registry, manually creating folders / files or similar activities). The major features of the application must have some functionality (i.e. in case of an email tool trying to send an email message should not crash the program)

For the application under test to be CERTIFIED by Quality Engineering as having passed Alpha, there must be:
o o o o o

NO P1 bugs No P2 bugs unless deferred by the project management team. P1 & P2 bugs have been verified. The application is feature complete (feature freeze) The pre-planned test suites were executed on frozen Alpha build (no changes made on the Alpha build to be certified).

William Gunkel

Page 11 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

For the application under test to be CERTIFIED as a Beta candidate by

Quality Engineering o o o

Code is frozen All issues identified during Alpha testing must be fixed and verified For purposes of Beta testing, the test environment must closely simulate the finally operational environment planned when the application is release, or be testing in the final environment

Internal Beta should as closely as possible simulate the finally operational environment planned when the application is release

William Gunkel

Page 12 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

For the application under test to be CERTIFIED by Quality Engineering as having passed Beta
o o o o

NO P2 bugs Code freeze in place No P3 bugs, unless deferred by the project management team P4 bugs will be reviewed at the release gate for possible impact on the customer experience and handled appropriately as decided by the project management team. P1, P2, P3 have been verified Pre-planned test suites were executed on frozen Beta build (no changes made on the Beta build to be certified) Developer documentation reflects state as tested to during Alpha certification process.

o o

William Gunkel

Page 13 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

For the application under test to be CERTIFIED by Quality Engineering to be ready for Golden Master CD
o

All product defects, regardless of their severity, have been fixed, documented, or otherwise resolved by the project management team. Project teams unanimous belief in readiness OR If Quality Engineering has not certified the product, the reasons while product was not certified must be stated in written report. The Program Manager in a written report will state the reasons why the product will be released without Quality Engineering Certification

o o

For the Golden Master to be certified for release nominal requires testing period of 1 week. NO defects are allowed at this stage

William Gunkel

Page 14 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Priority
(highest to lowest)

Description
Any issue that causes a catastrophic loss of application usability or loss of user data.

Examples of P1 issues are:

Priority 1 Any kind of application crash or lock up. Application installation failure Loss of data that can not recovered

Any issue that causes a serious reduction in performance or functionality of the application.

Example of P2 issues are:

Priority 2 Not supporting global user settings (user change of standard font, region etc. settings). PC based installation effects other already installed applications UI that dose not repaint correctly after updated or when first painted UI layout that dose not fit the required minimum screen resolution Major functionally missing that is required prior to Beta Release Loss of configuration settings or data (data loss can be recovered) Poor performance (very slow execution of commands poor or no user feed back while executing user commands) No feed back when supporting components or systems of the application are unavailable (which seriously effect the functionality of the application)

William Gunkel

Page 15 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Priority
(highest to lowest)

Description
Any issue that affects the users experience using the application, but does not stop the user from completing the operation attempted.

Example of P3 issues are:

Priority 3 Hidden or unclear command functions. Lack of feed back when supporting components or systems of the application are unavailable (which only has a minor effect on the functionality of the application). Program does not uninstall correctly Missing or wrong help Minor Feature not working Instructional Text which incorrectly describes the product usage and functionality Loss of focus or focus not were it is expected. Broken tab order (or lack of). Non-standard or missing short cut functions.

Any issue does not affect the users ability to use the application as designed.

Example of P4 issues are:

Priority 4 Typos & minor grammar,. Minor variations from industry UI standards (special use of short-cut keys)

William Gunkel

Page 16 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Reporting
o Bugs will be reported in TeamTrack. http://teamtrack.home.net/ Login and password can be obtained through Cecila Mateus: cecilia@excitehome.net Project Manager -> Thanh Lim QA Project Manager -> Bill Gunkel

William Gunkel

Page 17 of 18

12/5/2001

Excite@Home Cheetah Test Plan

Excite@Home Confidential

Revision History
March 26 2001 Updated the test configuration table

April 26, 2001 Added Bug Priority & Gate Standard section

William Gunkel

Page 18 of 18

12/5/2001