Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

Enterprise Delivery System Program Customer Portal

VDSS Master Test (SIT and UAT) Plan

Initial Draft Version 0.1

Project Name: Commonwealth of Virginia Department of Social Services Enterprise Delivery System Program Customer Portal System Test Plan Prepared By: Date ST Plan Prepared: Test Scripts Created By: System Testing Performed By: LDSS Staff Stephanie Napper November 10, 2011 DIS, Security, and Business Staff of DIS, Security, Business, and

v 0.1 11/09/2011

Page 1 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

Table of Contents
1. Introduction ......................................................................................... 4
1.1 1.2 1.3 1.4 1.5 1.6 Acronyms .......................................................................................................... 4 Definitions ......................................................................................................... 5 Purpose............................................................................................................. 5 Scope ................................................................................................................ 6 System Testing ................................................................................................. 6 Assumptions/Dependencies/Pre-Requisites/Risk ........................................... 11

2. Test Approach................................................................................... 12
2.1 Test Approach................................................................................................. 12 2.1.1 Requirement Testing................................................................................ 12 2.1.2 Regression Testing .................................................................................. 12 2.1.3 Interface Testing ...................................................................................... 12 2.1.4 Information Security Testing .................................................................... 13

3. System Testing Process .................................................................. 13

3.1 3.2 3.3 3.4 3.5 3.6 3.7 Test Entry/Exit Criteria .................................................................................... 13 Test Pass/Fail Criteria ..................................................................................... 13 Test Suspension/Resumption Criteria ............................................................. 14 Test Deliverables ............................................................................................ 14 Test Execution ................................................................................................ 14 Staff Responsibilities ....................................................................................... 10 Draft Schedule ................................................................................................ 15

4. Defect Tracking and Resolution Process........................................ 14 5. Communication........................................Error! Bookmark not defined.

5.1 Communication Process ................................................................................. 16

6. Change Management ........................................................................ 16

6.1 Changes to Master Test Plan.......................................................................... 16

7. Plan Approvals .................................................................................. 16

7.1 Changes to Master Test Plan.......................................................................... 16

v 0.1 11/09/2011

Page 2 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

Revision History

Version # 0.1

System Test Document: Revision History Date Description Author 11/09/2011 Initial Draft Stephanie Napper

Approvals

Department

Name and Title

Approval Date

v 0.1 11/09/2011

Page 3 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

1. Introduction
An examination of the Virginia Social Services System (VSSS) business model through business process re-engineering (BPR) revealed several issues and operational challenges within the state office and 120 local departments. Minimal customer self-service capabilities, paper and manual based processes, and limitations on information sharing among social services activities and partners were identified as operational challenges requiring a more progressive resolution. It was determined that these concerns could best be addressed by leveraging of more current technology. These concerns led to the inception of a new business process model; an efficient, state of the art, enterprise-wide VSSS servicing customers holistically. The Virginia Department of Social Services (VDSS) has entered into a partnership with Deloitte to create a web-enabled system that will move VSSS to service integration, improved efficiency, and improved outcomes for the citizens of Virginia.

1.1 Acronyms
The following acronyms may be found within this document. Acronym ACH ACS ADAPT BA BPR CP DIS EAL EDSP GIS IVR LDSS MMIS RQM SOW SIT SNAP SPIDeR TANF UAT UCD Description Automated Clearing House Affiliated Computer Systems, Inc. Application Benefit Delivery Automation Project Business Analyst Business Process Re-engineering Customer Portal Division of Information Systems Enterprise Audit Log Enterprise Delivery System Project Geographic Information System Interactive Voice Response Local Department of Social Services Medicaid Management Information System Rational Quality Manager Statement of Work System Integration Testing Supplemental Nutritional Assistance Program Systems Partnering in a Demographic Repository Temporary Assistance for Needy Families User Acceptance Testing Use Case Document

v 0.1 11/09/2011

Page 4 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

Acronym URD VaCMS VDSS

Description User Requirement Document Virginia Case Management System Virginia Department of Social Services

1.2 Definitions
Pre-Condition- The state in which the application must be before the test case is run. This may include pre-building test data prior to running a test. Post Condition- The state in which the application must be after the test case is run. If the post condition is met, the test passes. Test Case- Specific test record to be utilized in a defined test scenario and script (e.g. specific case/ component, known input, etc.). Test Scenario- High-level description of a test to be conducted (e.g. set of conditions, variables, positive/ negative scenarios, circumstance/ situation within which a case/ component is being processed, etc.). Test Script- The detailed steps/ instructions provided to the testers on how to operate the application and execute a test scenario to ensure it works according to the approved requirements. It contains a sequence of events and expected results. Application Requirement- The specific requirements the developers must program in order to meet the overall requirements of the test scenario.

1.3 Purpose
The purpose of this Master Test Plan document is to define and outline processes, activities, tools, and responsible parties for the System Integrated Testing (SIT), Security, and User Acceptance Testing (UAT) of the Enterprise Delivery System Project - Customer Portal (EDSP CP). This form of testing is to validate that Customer Portal, ADAPT and all interfaces meet documented requirements as stated in the following documents: Statement of Work (12/10/10) Business and Requirement Document Use Case Documents Development Clarification Meeting Documents

v 0.1 11/09/2011

Page 5 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

1.4 Scope
The System Integrated Testing (SIT) jointly performed by Deloitte, and the Division of Information Systems (DIS) will be followed by User Acceptance Testing (UAT) performed by Security and Business. The test schedule below outlines the two phases: Release 1 Orientation and Training for UAT (Magic Team) November 8 9, 2011 SIT/MIT Developing of Test Scripts (11/18/2011) SIT Training for testers (regular ongoing meetings) Test Execution: MIT/SIT (November 21, 2011 January 27, 2012) UAT Developing Test Scripts (December 6 14, 2011) UAT Building Cases in ADAPT (December 15, 2011 January 12, 2012) UAT Orientation for LDSS Testers (January 18, 2012) UAT Introduction to Test Scripts and new ADAPT screens (January 19, 2012) Test Execution : UAT - Business and Security (January 24 March 1, 2012) Release 2 Test Execution: SIT ( March 12 May 4, 2012 Orientation and Introduction to Renew, Changes, and Check My Benefits for Testers (May 2 3, 2012) Test Execution: Business and Security (May 7 June 7, 2012)

1.5 System Testing

This System Test Plan document covers the methods of testing used to verify that the Enterprise Delivery System Project Customer Portal delivered by Deloitte meets the system requirements outlined in the Statement of Work, Requirement Documents, Use Case Documents, and Development Clarification minutes. This document describes the comprehensive approach to system testing, including specific areas of the system that will be tested, areas that will not be tested, the roles of the individuals involved in the testing, the documentation involved in testing, and the procedures for reporting and retesting defects found during this phase of testing. Testing of the system will be predicated upon the releases for this project. Release 1: This phase of testing will focus on all aspects SIT, Security, and UAT for the Screening, Apply for Benefits, and My Work Space modules of the application for TANF, SNAP, Medical Assistance, and Energy Assistance. Testing for this deployment to include SIT, Security, and UAT will be conducted from 11/21/11 through 3/1/12. Release 2: This phase of testing will focus on all aspects; SIT, Security, and UAT for the Check My Benefits, Report Changes, and Renew My Benefits modules of the applications for SNAP, TANF, Medical Assistance, and Energy Assistance.

v 0.1 11/09/2011

Page 6 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

MIT/SIT Definition: MIT includes initial testing of the individual components/ modules and their preliminary integration into the solutions back-end framework. MIT will be independent of the front-end solution and will focus on the back-end solution and related components. SIT includes initial testing of the individual components/ modules and their preliminary integration into the solutions back-end framework. MIT will be independent of the front-end solution and will focus on the back-end solution and related components. SIT takes an optimistic approach to testing, because it expects to have few problems with the individual components due to Unit Testing and MIT. The strategy relies heavily on the component developers to perform the isolated unit testing for their product before being released to MIT and then SIT. The goal of the strategy is to avoid redoing the testing done by the developers and initial MIT testers and instead determine problems caused by the interaction of the components in the environment. SIT includes system to system and interface to interface testing. It includes the testing of the integration of the front-end solution with the back-end solution. To be more efficient and accurate, user-like workloads (e.g. for customers, for workers) will be defined for creating realistic scenarios in exercising the environment. This allows for confirmation that the integrated environment will work as expected for the target customers/ workers. Participants: IT Test Manager Mary Disse IT Requirements Manager Iesha Tiller 4 Business Analysts Jan Ellison Fernando Miller Willie McWhite John Vaughan Method: MIT/ SIT will conduct requirements-based testing focusing on the following test scenarios and their associated requirements: Security (e.g. including authentication, authorization, associations, EAL and SAMS Reports) Apply For Assistance through Customer Portal (CP) Interface o Including Reapply for assistance through CP

v 0.1 11/09/2011

Page 7 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

Renew My Benefits through CP Interface o Including Add person or program with renewal of benefits through CP Report My Changes through CP Interface, including: o Address change o Add a person o Remove a person o Add a program o Begin/end pregnancy or disability Check My Benefits through CP Interface (including Application Status checks) File Clearance through My Workspace (MWS) and ADAPT FIPS Verification through MWS (using GIS) Reference Table Validations (ADAPT and CP) Changes to ADAPT required to accommodate the above scenarios (e.g. including errors, screen help, alerts, inboxes, summary screens, reports, etc.).

Test scenario documents, requirements traceability matrices, screen mockups, design documents, and help files in the form of the CP/ADAPT VDSS Back-End Design Document and supplemental documents/ data matrices will be provided to testers. The Test Scripts which include steps the testers will follow to test the scenarios and their requirements will be developed by and provided to all testers.

Security Definition: testing encompasses the back end of this project as it relates to both citizens and staff of local departments of social services entering and accessing secure information, to include creating accounts, strong passwords, resetting passwords and access to information is provided as associated with client IDs.. Separate tests will be conducted on the SAMS CP functions with existing Security Staff. An established SAMS regression test process will be utilize as part of these tests. Separate test account for each worker role in CP will be tested. COV Auditors will assist in conducting tests on the CP. Participants Test Manager Matt Teasdale Existing Security Staff 10 COV Auditors Method: Security will conduct requirements-based testing. Each tester will enter data and log the inputs as they make them for comparisons with the audit logs. Testers will purposely attempt to enter values that should be rejected. Testers will process file clearance actions to see how they are logged in EAL.

v 0.1 11/09/2011

Page 8 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

Testers will add, update and delete accounts to see how the system responds and updates to EAL. Testers will create a CP account and test the password reset options. Separate tests will be conducted on the SAMS CP functions with existing Security Staff.

UAT Definition: Business UAT as are all components of testing has a primary expectation to test with the goal of releasing to the users; both the citizens of the Commonwealth and DSS staff an efficient new technology that will enhance the way we do business the first time out of the gate. To that extent UAT focuses on the testing of business rules for SNAP, TANF, Medical Assistance, and Energy Assistance utilized in the EDSP CP and the ADAPT System to ensure that programming of rules and logic result in accurate outcomes in line with the Requirements Document. UAT includes testing of navigational tools, password generation, access to client information, system and screen flow, the accuracy of the passing of information from the Customer Portal to ADAPT to include the filing of an application utilizing the Spanish language, and the generation of reports in My Work Space. UAT includes ease of use testing through the partnering with The Virginia Department for the Blind and Vision Impaired. Participants: Co-Test Managers Stephanie Napper Delores Veal 3 Subjects Matter Experts (Regional Consultants who have been with the project from the first JAD session) Sherry Sinkler-Crawley Nan Foster Cathy Trimble 2 Testers from local departments of social Services Gail Kelley Norfolk Department of Human Resources Julie Chalupsky - Norfolk Department of Human Resources Possible Testers from VDSS Business Operations Unit Method: UAT will conduct requirements-based testing conducted in a test lab. All testers will test from this lab. The lab environment will provide computers for each tester. Orientation will be held for all testers to provide guidance for testers who will develop test cases and test scripts. An additional orientation will be held for all testers who will perform system testing. Testers developing scenarios and

v 0.1 11/09/2011

Page 9 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

scripts will be provided access to Use Case Documents, hard copy of the VDSS Back End Design Document, and the Requirement Document. Testing of the Customer Portal; Screening, Apply for Benefits, My Work Space, Filing Renewals, Reporting Changes, and Check My Benefits will include: Accuracy of information on screens Input of information Expected results vs. Actual results Testing of Requirements Test to ensure they cannot get off the screen without a strong password Test to ensure that only the identified individual(s) can access case information

Testing of the ADAPT System to ensure information pushed from the Customer Portal to ADAPT is correctly populated in the appropriate subsystems and that no facet of eligibility determination and benefit calculation is automatically ran. LDSS testers will have primary role in ensuring that screens other than identified screens have not changed. LDSS tester will have primary role in ensuring that the system functions the way it did prior to the new application other than identified changes. Testers will add program to both pending applications and ongoing cases. Testers will add/delete a person to both pending applications and ongoing cases.

1.6 Staff Responsibilities

VDSS/DIS/Security/Business staff will be responsible for creating the System Test Plan and creating and maintaining System Test Case Scripts using the documents noted in above sections. The assigned staff will be responsible for securing test documents and securing production data used for testing. Test Managers are responsible for the overall environment, operations, and process of the units system testing. Test managers will review and key defects into ClearQuest. Test Managers will monitor the lifecycle of defects. Testers are responsible for accurate and precise testing of test scripts and regression testing as needed. Testers are also responsible for reporting defects to Test Managers. Testers will maintain a daily count of test scripts tested to include pass/fail results.

v 0.1 11/09/2011

Page 10 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

1.7 Assumptions/Dependencies/Pre-Requisites/Risk
Assumptions The following assumptions have been made for this plan: Deloitte, DIS, Security, and Business will commit adequate resources to support the testing effort as listed in the project schedule. All components of testing (Deloitte, SIT, Security, and Business) All testing will be manually performed. Testers, Test Managers, and Project Co-Leads will be made aware of all defects, new and outstanding. Review of defects will be review by Test Managers to ensure the defect is valid, supporting documentation is provided to explain defect, and that an identical defect has not been previously submitted. Defects will be corrected in a timely manner to ensure the success of the project schedule. Regional and LDSS personnel participating in testing understand 100% of their effort is required. Deloitte, MIT/SIT, Security and UAT agree upon the severity definition for defects identified during testing. Deloitte, MIT/SIT, Security and UAT will have access to defects created in ClearQuest. All defects will be entered in ClearQuest daily by 3:00 p.m. Defects with a critical or high severity level will also be called in to the identified person for Deloitte and DIS. Dependencies and pre-requisites The following dependencies and prerequisites exist: Defects found during SIT have been resolved before UAT can begin. All testing resources (computers, requirements, SOW) must be in place. All test environments (ADAPT, SPIDeR, MMIS, VaCMS, Energy Assistance Automated Program, LDSS interface, etc) must be properly established Test environment matches real world Testers are experts in their own area Testers from LDSS are knowledgeable of all benefit programs All defects found during testing have been resolved to an agreed upon acceptable level.

Risks The following risk may exist: Resources not being available when needed to test Users and/or developers trying to make changes to previously approved requirements Findings on pre-existing production issues and requests for resolutions (potential scope creep). Non-CP related legacy system or interface changes impacting this project.

v 0.1 11/09/2011

Page 11 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

2. Test Approach
2.1 Test Approach
System testing will be performed manually for SIT, Security, and UAT. Test scripts will be executed using valid and invalid data to verify the expected results occur. Each test script or test case will be created, tested, and documented as pass or fail. Test scripts may be combined into one document or maintained as individual documents with the goal of entering all test scenarios and scripts in Quality Manager. The IT, Security, and Business Test Managers should be abreast of all test scenarios and scripts. Each test case/script will be traced to a Business Requirement. There may be rare situations where a test can not be located in the Business Requirements because discrepancies may exist. While the Rational Suite, including Rational RequisitePro, ClearQuest, and Quality Manager, are the systems of record for the EDSP-CP project a back-up method will be implemented at the beginning of the process. All requirements are loaded and managed in RequisitePro; all defects must be logged/ tracked in ClearQuest, and all test items will be included in Quality Manager. This will ensure traceability and consistency throughout the entire project lifecycle. Test Cases and Test Scripts are to be written and compiled on the agreed upon format for consistency. Test Cases and Scripts will include a unique numbering identifier, the associated requirement number(s), and a pithy description of the intent of the scenario and script. Paper documents will be electronically filed in a determined folder on the R: / drive.

2.1.1 Requirement Testing Each applicable numbered requirement will be traced to a test script/case for traceability to validate each function properly works and meets the needs of the user. 2.1.2 Regression Testing MIT/SIT, Security, and UAT will each conduct regression testing. Regression testing will be performed on defects introduced during system testing only. Regression testing will also will include testing of existing legacy system functionality to verify that it was not impacted unexpectantly when other aspects of the system/interface were changed to meet project requirements. 2.1.3 Interface Testing MIT/SIT will test not only the individual components/requirements, but will also test the interfaces between the CP and ADAPT/ MMIS/ Energy system/ LDAP/ SPIDeR/ EAL/ GIS, both by module and by end to end process. This includes the CP data, ESB translations, VDSS Holding Tables, and legacy system databases. This is to ensure that the data entered into the CP is accurately transmitted and displayed in ADAPT. Also, it

v 0.1 11/09/2011

Page 12 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

will test that the data in ADAPT/ MMIS/ Energy system is accurately transmitted and displayed in the CP during the Renew My Benefits, Report My Changes and Check My Benefits processes. 2.1.4 Information Security Testing Information security testing will be performed differently because of its complexity and the number of roles versus groups versus user types. Testing will consist of validating system access and allowing of information to clients through the CP.

3. System Testing Process

3.1 Test Entry/Exit Criteria
System Testing will begin after tester training has been completed, test case scripts documentation have been initiated and approved in draft, and system access and logon have been obtained for the testers from SBE. All resources must be in place and all test environments must be properly established and connections available. System Testing will be considered complete when all planned test scripts have been executed, all known system testing defects have been documented, tracked, and corrected defects have been re-tested with no outstanding Critical or High defects or at the end of the contract period, whichever comes first. A critical defect is a defect that prevents further testing. A high defect is a defect that that does not functions as expected/designed or causes other functionality to fail to meet requirements. Otherwise, system testing will be considered complete on March 2, 2012 for Release 1 and June 8, 2012 for Release 2 per the current Statement of Work Contract.

3.2 Test Pass/Fail Criteria

The pass/fail criteria for each test case/script is described by its expected results and will be noted as pass or fail (with a defect number assigned), the test date, and tester name. If the expected results are received, the test will be marked as pass, if the expected results are not received, the test will be marked as fail. Upon defect correction, the test case/script will be retested and again noted as pass or fail during retest. Each test case/script will continue to be retested until it passes or is deemed acceptable by the EDSP CP Business Manager.

v 0.1 11/09/2011

Page 13 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

3.3 Test Suspension/Resumption Criteria

If a problem is found that impedes the continuation of any phase of testing (50% or more of the test cases can not be completed), System Testing will be suspended until appropriate corrections are made and testing can resume. If this type of suspension is warranted, the EDSP CP Test Manager will immediately be notified.

3.4 Test Deliverables

The following items are work products that will be outputs of the testing process. System Test Plan System Test Case Scripts that includes a traceability matrix

3.5 Test Execution

System Testing will be performed in a test environment at VDSS by testers identified by DIS, Security, and Business. A stable URL will be provided by Deloitte. Testers will have access to the unit Test Manager on a daily basis (in person, phone or email) during the testing period to discuss issues and questions. Workstations will consist of the hardware setup used by VDSS within the department and access to the test environment will be secure and only accessed by the individuals tasked with this activity. Testing will be executed manually by using test case scripts that will be captured in MS WORD or MS Excel Spreadsheets. Each script will contain the object of the test, the conditions to perform the test, the expected results, the actual results received, the testers name, the date of test, and the execution status of the test (pass/fail). The execution of test scripts are governed solely by the tester assigned and can be executed in any order deemed necessary by the tester. The test database will consist of production and test data; therefore, data will be handled securely: Production data will not be accessed nor viewed by anyone in Deloitte or VDSS/DIS.

4. Defect Tracking and Resolution Process

Any defects encountered will be reported by the testers at the end of the day, with the exception of Critical or High defects and entered directly into ClearQuest. Critical and High defects are to be reported to the Test Manager immediately. Defect tracking and resolution will occur at the test script level. System testers will execute the test scripts and record within the script the step at which the script failed. If a script fails, it will be recorded within the defect tracking system and attachments will be supplied as evidence.

v 0.1 11/09/2011

Page 14 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

The following table describes the Defect Severity levels: Severity Level Critical High Definition A defect that prevents further testing of the system, A defect that does not function as expected/designed or causes other functionality to fail to meet requirements. A defect which does not conform to standards and conventions. The application or system may function properly, but the actual test results are not the same as expected results. (Should be evaluated on a defect by defect basis.)

Medium

Cosmetic defects which does not affect the functionality of the application or system. It is expected that 100% of all defects be resolved. Low ClearQuest statuses will be used to track the defect through the testing and retesting processes. This status will be used to meet project reporting needs.

5. Schedule
5.1 Draft Schedule
The following dates are estimates based on the project plan and deadline date for completion of system testing. SIT Release 1 Completed: Security and UAT Release 1 Completed: SIT Release 2 Completed: Security and UAT Release 2 Complete: January 27, 2012 March 2, 2012 May 4, 2012 June 8, 2012

v 0.1 11/09/2011

Page 15 of 16

Virginia Department of Social Services Enterprise Change Management

Master Test Plan EDSP - CP

5.2 Communication Process

A weekly communications meeting with Deloitte, DIS, Security and Business will be held to discuss status of the system testing effort, status of interface and web services, and outstanding items that affect system testing. Deloitte should also be available to for additional meetings and phone calls as needed. Defects identified by a VDSS team member that are determined to be related to Deloitte requirements will be submitted to the teams Test Manager who will submit the defect to the Deloitte PM in ClearQuest. The VDSS Tester will notify the appropriate Test Manager, and the Test Manager will notify the Deloitte PM by email of the entry in ClearQuest. Such defects will be linked/ traced to the Deloitte Requirements as entered in RequisitePro. The Deloitte PM will then assign the defect to the appropriate Deloitte technical lead for resolution. Upon resolution, the Deloitte PM will re-assign to the IT Test Manager for any required review or retesting. Deloitte identified defects that are determined to be related to VDSS requirements will be assigned by the Deloitte team member to the appropriate Test Manager in ClearQuest. The Deloitte team member will notify the Deloitte PM, and the Deloitte PM will notify the IT Test Manager by email of the entry in ClearQuest. The IT Test Manager will work with the Technical Leads to assign the defect to the appropriate DIS team member for resolution. Upon resolution, the IT Test Manager will re-assign to the Deloitte PM for any required review or retesting.

6. Change Management
6.1 Changes to Master Test Plan
Changes to the Master Test Plan will be made by Stephanie Napper or Delores Veal, Co-Test Managers. Test Managers for SIT and Security should submit requested changes via email.

7. Plan Approvals
7.1 Changes to Master Test Plan
The Master Test Plan and any changes will be approved by the Project Co-Leads.

v 0.1 11/09/2011

Page 16 of 16