DSB BMS 201801 January Monthly QA Report #5 v1.1

Washington State
Department of Services for the Blind
Business Management System (BMS)
QA Monthly Report #5 – January 2018

(for the period of 12/20/2017 – 2/1/2018)
V 1.1
2/1/2018
Prepared By:
CASE
Associates
Incorporated
Dan Kruger, Project Lead, Quality Assurance Consultant

Jamie PoVey, CPM, Senior QA Analyst
Contract # 430000266
Department of Services for the Blind BMS Project – January 2018 QA Monthly Report #5
Document Revision History
Version Date Comments

Submit Draft report to DSB PM and Executive
1.0 1/29/2018
Sponsor
Receive comments and incorporate changes if
1.1 2/1/2018 necessary and post the final report to OCIO
Project Website.
Prepared by CASE Associates Inc. Page 2 Contract # 430000266

Table of Contents
Document Revision History ............................................................................................. 2
Table of Contents .............................................................................................................. 3
Executive Summary .......................................................................................................... 5
Overall Project Progress ...........................................................................................................5
Major Risks Identified During the Current Monthly Assessment Process .........................5
Overall Project Health ...............................................................................................................6
Overall Project QFP Status – Stable ........................................................................................6
QFPs rated as Stable .................................................................................................................6
QFPs rated as Needs Attention ................................................................................................7
QFPs rated as needing Immediate Attention..........................................................................7
QFPs Not Rated – NR ................................................................................................................7
Quality Focal Point (QFP) Summary Chart .............................................................................7
Recommendations for QFPs Rated Stable (Stable) ................................................... 10
Recommendations for QFPs Rated Needs Attention (Attention) ............................. 10
Recommendations for QFPs Rated Alert (Alert) ......................................................... 10
Detail Quality Focal Point Assessment ........................................................................ 11
QFP 1 - Project Phase Readiness/Completion ..................................................................... 11
QFP 2 - Requirements Management ...................................................................................... 12
QFP 3 - Project Schedule ........................................................................................................ 13
QFP 4 - Communications ........................................................................................................ 14
QFP 5 - Risk and Issue Management..................................................................................... 15
QFP 6 - BMS Vendor Performance ........................................................................................ 16
QFP 7 - Technical Transition, Business Process Re-Engineering .................................... 16
QFP 8 - Project Organization and Leadership ...................................................................... 17
QFP 9 - Project Resources ..................................................................................................... 19
QFP 10 – Project/Quality Management and Reporting........................................................ 19
QFP 11 - Budget Planning and Tracking............................................................................... 20
QFP 12 - Scope and Change Control..................................................................................... 21
QFP 13 – Roles, Responsibilities, and Communications ................................................... 22
QFP 14 - IT Architecture .......................................................................................................... 22
QFP 15 - IT Acquisition Management .................................................................................... 23
QFP 16 - Project Library and Configuration Management .................................................. 23
QFP 17 - System Definition Process ..................................................................................... 24
QFP 18 - System Design Process .......................................................................................... 25
QFP 19 - Data Conversion/Migration ..................................................................................... 25
QFP 20 – Configuration/Construction Process.................................................................... 26
QFP 21 - Testing (Functional, Capacity, and Performance) ............................................... 27
QFP 22 - User Acceptance, Business Process Transition ................................................. 28
QFP 23 – Training .................................................................................................................... 28
QFP 24 - Inter/Intra-Agency Coordination and Implementation ......................................... 29
QFP 25 - Implementation Process ......................................................................................... 29
QFP 26 - Deployment Process ............................................................................................... 30

QFP 27 – Interface Design and Implementation................................................................... 31

Appendix A – Project Activities, Meetings, Deliverables........................................... 32
Meetings Attended: ................................................................................................................. 32
Documents Reviewed: ............................................................................................................ 32
Documents / Reports Delivered: ............................................................................................ 33
Documents / Reports / Meetings for Next Month:................................................................ 33
Appendix B - QA Methodology and Report Structure ................................................ 34
Purpose ..................................................................................................................................... 34
Methodology ............................................................................................................................. 34
Appendix C – QA Recommendations and Response Tracking ................................ 36
Open QA Recommendations .................................................................................................. 36
Closed QA Recommendations and Agency Responses..................................................... 36

Executive Summary
CASE Associates Inc. (CAI) has completed the January 2018 monthly QA report #5 for the
period from December 20, 2017, through January 31, 2018. The project has been in full
implementation for two months. The project has completed the GAP analysis for all the
program areas (BEP, VR, Fiscal and OTC) and completed all AWARE system training for
Subject Matter Experts (SMEs) before initiating VR, Fiscal and OTC adaptation configuration
sessions.
The vendor (Alliance) submitted the fully resourced project schedule to the project team in
December. The DSB PM has made the necessary changes to this schedule to meet DSB’s
project goals. The project schedule was approved by the BMS Steering Committee on January
25, 2018.
The project team has prepared the required documents for the OCIO approval for IT Pool
implementation monies for the project. The initial review has been completed by the OCIO and
final approval is expected very soon.
CAI performed the following activities during the monthly reporting period to determine the
Business Management System (BMS) Project’s health:
 Participated in project team meetings to observe, review, and discuss project progress
and issues.
 Participated in the January 2018 Steering Committee Meeting.
 Attended and observed the OTC AWARE training session in the Seattle office and made
recommendations for improvement to the Alliance trainers.
 Reviewed the project SharePoint site project documents and many of the Alliance
deliverables.
 Participated in a conference call with the OCIO oversight representative (KPR) to review
the project status. Also met with the new OCIO oversight consultant (Whitney
Dickinson) to provide her the major QA findings to date.
Overall Project Progress

The project continues to be well managed and the project’s key deliverables from the vendor
are ahead of the originally planned milestones. The BMS project team continues to identify
project risks and issues, review proposed vendor milestone dates and has provided feedback
on the draft integrated project schedule. The project schedule was approved by the BMS
Steering Committee on January 25, 2018.
Communication strategies are being implemented and the project is being accepted by the
stakeholders. The project team has completed the GAP analyses for the project (BEP, VR,
Fiscal and OTC). BEP adaptation sessions were completed in January and the final
recommendations for BEP configuration were completed. January was a major training period
for the project. AT, OTC and VR training was completed during this reporting period. The VR
adaptation process begins the last week of January 2018.
Major Risks Identified During the Current Monthly Assessment Process

There are no major risks in the project at the writing of this report.

Overall Project Health

The overall health of the project is Stable based on the three primary areas described in the
table below:
Health Factor Rating Comments
The project scope has been fully defined by the project team
Scope Stable and Alliance. The Project Sponsor and Project Team have
worked closely with Alliance to accurately identify the final
scope of deliverables.
The BMS Steering Committee approved the project schedule
on January 25, 2018. The BEP business area is scheduled to
Schedule Stable go first into production in April 2018. Currently, there is little
slack-time in the approved project schedule. CAI is not
concerned at this time since the project scope is well defined
and stable.
DSB will be submitting the application for Implementation
Funding from the IT Pool in late January or early February
Budget Stable 2018. The project team has identified potential cost savings by
eliminating several customizations that may not be essential in
meeting the business need. These savings can fund other
project areas within the current scope of the project.
Overall Project QFP Status – Stable

CAI’s assessment in this section is based on reviewing BMS’s management processes and
artifacts using the Quality Focal Points (QFP) that are applicable during this stage of the project.
Each QFP is a specific assessment of the project’s overall planning and execution of the BMS’s
management plans and incorporated processes. QFPs are rated as “Stable” (Green), “Needing
Attention” (Yellow) or “Alert” (Red - meaning that the QFP requires immediate attention).
QFPs rated as Stable

In this report, 19 QFPs were rated as Stable:
 QFP 1: Project Phase Readiness/Completion
 QFP 2: Requirements Management
 QFP 3: Project Schedule
 QFP 4: Communications
 QFP 5: Risk and Issue Management
 QFP 6: BMS Vendor Performance
 QFP 7: Technical Transition (T) and Business Process Re-engineering (B)
 QFP 8: Project Organization and Leadership
 QFP 9: Project Resources
 QFP 10: Project/Quality Management and Reporting
 QFP 11: Budget Planning and Tracking
 QFP 12: Scope and Change Control
 QFP 13: Roles, Responsibilities, and Communications
 QFP 14: IT Architecture
 QFP 15: IT Acquisition Management

 QFP 16: Project Library and Configuration Management

 QFP 17: System Definition Process
 QFP 23: Training
 QFP 24: Inter/Intra-Agency Coordination and Implementation
QFPs rated as Needs Attention

In this report, no (0) QFPs were rated as Needs Attention:
QFPs rated as needing Immediate Attention

In this report, no (0) QFPs were rated as Alert:
QFPs Not Rated – NR

In this report, 8 QFPs were not rated as they assess project activities that will occur later in
the project:
 QFP 18: System Design Process
 QFP 19: Data Conversion/Migration
 QFP 20: Configuration/Construction
 QFP 21: Testing (Functional, Capacity, Performance)
 QFP 22: User Acceptance, Business Process Transition
 QFP 25: Implementation Process
 QFP 26: Deployment Process
 QFP 27: Interface Design and Implementation
Quality Focal Point (QFP) Summary Chart

Projects are dynamic and QFP ratings can change their status from month to month. When that
occurs, the report will describe the events leading up to the change in the monthly status and
the impacts to the project schedule. The table below provides a running multi-month view of the
27 Quality Focal Points and the projected next month’s rating.
Monthly Quality Focal Point Assessment (QFP) Assessment

#1 #2 #3 #4 #5 Trending
Quality Focal Point Aug Sep Oct Nov Dec/Jan Indicator
1
(QFP) Assessment Impact 2017 2017 2017 2017 2018
1. Project Phase Stable Stable Stable Stable
High Stable
Readiness/Completion
2. Requirements NR Stable Stable Stable
High NR
Management
3. Project Schedule High Attention Attention Attention Attention Stable
4. Communications High Stable Stable Stable Stable Stable
1
Expected assessment results for the next reporting period (next month)


#1 #2 #3 #4 #5 Trending
1
5. Risk and Issue Stable Stable Stable Stable
High Stable
Management
6. BMS Vendor NR Stable Stable Stable
High NR
Performance
7. Technical Transition Stable - Stable - Stable -
(T) and NR - T NR - T T T T
High
Business Process Re- NR - B NR - B Stable - Stable - Stable -
engineering (B) B B B
8. Project Organization Stable Stable Stable Stable
High Stable
and Leadership
9. Project Resources High Stable Stable Stable Stable Stable
10. Project/Quality
Management and Med Stable Stable Stable Stable Stable
Reporting
11. Budget Planning and NR NR Stable Stable
Med NR
Tracking
12. Scope and Change Stable Stable Stable Stable
Med Stable
Control
13. Roles,
Responsibilities, and High Stable Stable Stable Stable Stable
Communications
14. IT Architecture High NR NR Stable Stable Stable
15. IT Acquisition NR NR Stable Stable
High NR
Management
16. Project Library and
Configuration Med NR NR Stable Stable Stable
Management
17. System Definition High NR NR Stable Stable Stable
Process
18. System Design High NR NR NR NR NR
Process
19. Data High NR NR NR NR NR
Conversion/Migration
20. Configuration/Constru High NR NR NR NR NR
ction
21. Testing (Functional,
Capacity, High NR NR NR NR NR
Performance)


#1 #2 #3 #4 #5 Trending
1
22. User Acceptance,
Business Process High NR NR NR NR NR
Transition
23. Training High NR NR NR Stable Stable
24. Inter/Intra-Agency
Coordination and High NR NR NR Stable Stable
Implementation
25. Implementation High NR NR NR NR NR
Process
26. Deployment Process High NR NR NR NR NR
27. Interface Design and Med NR NR NR NR NR
Implementation
The next month indicators signify expected changes in the QFP ratings.
- This Quality Focal Point (or group of QFPs) is expected to have the same
rating in next month’s Periodic QA Evaluation.
- The risk is increasing the Quality Focal Point. It may be changed to
Alert in next month’s evaluation
- The risk is decreasing on the Quality Focal Point. It may be changed to
Attention or Stable in next month’s Evaluation.
- The risk is increasing the Quality Focal Point. It may be changed to
Attention or Alert in next month’s evaluation.
Stable in next month’s evaluation.

Recommendations for QFPs Rated Stable (Stable)

No “Stable” recommendations were identified in the 5th monthly QA report. The project team is
functioning at a high-level and is tracking and monitoring the project schedule, tasks and
activities, and vendor deliverables appropriately.
Recommendations for QFPs Rated Needs Attention

(Attention)
No “Needs Attention” recommendations were identified in the 5th monthly QA report. The
project team was successful in getting the integrated project schedule approved by the Steering
Committee on January 25, 2018.
Recommendations for QFPs Rated Alert (Alert)

No “Alert” recommendations were identified in the 5th monthly QA report.

Detail Quality Focal Point Assessment

1 QFP
4 1 - Project Phase Readiness/Completion
Project phase readiness determines if the project is ready to move onto the next phase and will be successful at
completion. The Project Phase Readiness/Completion QFP assesses whether the overall project is complying with the
requirements within schedule and budget. Interim and major project milestones are assessed when using a traditional
Waterfall methodology.
Stable Attention Alert
Risk Level
1.1 Is the project complying with requirements? Findings: Yes. The project is fully in the implementation phase of the project.
CAI continues to observe that the project team has been clearly focused on moving the project forward at a rapid pace. The
project team and Alliance have completed the integrated project schedule which was approved by the project Steering
Committee on January 25, 2018.
1.2 Is schedule performance (e.g., planned/actual productivity or throughput in meeting requirements) being met so far?
Findings: Yes. The project managers from DSB and Alliance have a final detailed integrated project schedule with the key
major milestones, dates, and deliverables.
1.3 Is there sufficient time (with appropriate slack) to complete the project before the committed completion date? Findings: Yes.
Alliance and DSB PM’s have been consistently exceeding the dates in the working integrated project schedule. At this time
CAI does not see any evidence that this cadence will not be maintained. The production roll-out of the BEP module is
scheduled for April 2018. The remaining project system roll-out is scheduled for March 2019. CAI will be monitoring progress
on deliverables as the project proceeds.
1.4 Is cost performance (planned/actual costs in meeting requirements) being met so far? Findings: Yes. The project is
managing project costs and deliverables according to the signed contract. Over the next several months, CAI will be able to
do a deeper assessment of project cost performance.
1.5 Is there sufficient budget (with appropriate contingency) to complete the project before the committed completion date? Yes.
The full funding of the project has been secured through the State IT Funding Pool. The Project Sponsor is managing the
project to this budget. There has been constant review by the BMS project manager to monitor and control costs during the
project. After completing the GAP analysis for BEP, VR, and OTC and completing the BEP adaptation session, there appears
to be less effort required to implement the system. This is still early in the analysis of cost saving opportunities. CAI will be
monitoring this effort for cost savings and meeting project objectives. The project team is completing their application for
implementation funding through the OCIO for monies from the IT funding Pool.
Recommendations: Complete the IT Funding Pool application for implementation monies for the project.

2 QFP
5 2 - Requirements Management
The Requirements Management process is appropriate and thorough. The Requirements Management QFP assesses
the functional needs of the product and/or service performed by the business, including business process workflow. There
are various types of requirements but not limited to business and workflow, product, software, hardware, and security
requirements.
Risk Level
2.1 Are the Requirements understood and confirmed? Findings: Yes. All requirements have been identified during the contract
negotiations. The BEP, VR, Fiscal and OTC GAP analyses are complete. Changes to the requirements have been made but
no show-stoppers have been identified. In fact, there appears to be functionality in the AWARE product that can be used
without additional adaptation but more of a change in the business process. Only one major GAP was identified in BEP, but
the plan is to use AWARE analytics to resolve this gap. The project team is reviewing the Aware Analytics approach and
more will be known next month.
2.2 Are requirements traceable to design? Findings: Yes. This is a COTS product implementation and DSB requirements will be
configurable in the AWARE system. For those DSB requirements that are not part of the AWARE system, the agency has
been working with staff to develop appropriate workarounds for their business needs. This approach was discussed early on
in the project and all stakeholders have agreed to this approach.
2.3 Are requirement change impacts understood and documented? Findings: Yes. The project team has been clear throughout
the planning and initial stages of implementation that this is a COTS solution and the agency will not be making
“customizations” to this system. The Project Sponsor, PM, Business Analyst and the OCM leaders in this project have been
articulating this message from the initial discussions on the project.
2.4 Are test conditions defined to validate requirements compliance? Findings: Unknown at this time. Evaluation of this QFP
will occur in a future report as the implementation of the system becomes more mature.
Recommendations: None

3 QFP
3 3 - Project Schedule
The project is appropriately planned. The Project Schedule QFP provides an assessment of the breadth and depth of
project planning, scope definition, the scheduling of the tasks, including identification of internal and external dependencies.
Traditional methodology defines the schedule for the completing the entire project scope.
Risk Level
3.1 Are all appropriate tasks identified in the Work Breakdown Structure (WBS)/Project Schedule? Findings: Yes. The project
schedule was agreed to during contract negotiations. The BMS and Alliance PM’s have developed the integrated project
schedule that has been approved by the project Steering Committee.
3.2 Are dependencies among tasks identified, including decision dependencies? Findings: Yes. CAI has conducted our initial
review and the working integrated complete project schedule has the required detail WBS to meet the project demands.
3.3 Has a Project Schedule been established and is it reasonable based on resources (budget), productivity, assumptions, and
dependencies? Findings: Yes. The overall project schedule was negotiated during the contract signing. An integrated
project schedule was approved by the BSM project Steering Committee on January 25, 2018. The BMS PM is continually
reviewing and updating the BMS project schedule.
3.4 Is the Project Schedule detailed enough to monitor progress? Findings: Yes. The integrated project schedule provides the
required detail of the tasks, activities, and dependencies to track and monitor progress in the project.
3.5 Is the Project Schedule used to track progress and updated on a regular basis? Findings: Yes. The BMS project manager is
tracking percent completion and all tasks.
3.6 Are external project dependencies identified in the Project Schedule? Findings: Yes. The BMS PM and Business Analyst
have done an incredible job in identifying all dependencies in the project. As the project progresses, the project team is
carefully monitoring for any dependencies not previously identified, and if necessary develop a strategy for mitigation.
3.7 Have appropriate interim and major milestones been defined? Findings: Yes. The integrated project schedule has the
milestones, dates, and deliverables identified. Only the initial project activities that have been started are complete.
3.8 Has the Project Plan been reviewed, approved and signed off by the project Stakeholders/Product Owner? Findings: Yes.
The Integrated Project Management Plan is complete and was approved by the BMS project Steering Committee on January
25, 2018.
3.9 Is there an appropriate process for updating the Project Schedule with actuals and tracking project progress? Findings: Yes.
The Project Manager has a process for updating all project schedule updates on a routine basis.
3.10 Are reasonable plans available to manage the Project? Findings: Yes. The BMS Project Manager is Rolanda Maddox and
she is experienced and will be using the appropriate tools to manage the project. Her backup is Brent Carr.

Recommendations: None
4 QFP
5 4 - Communications
The project communications are effective and adequately controlled. The Communications QFP examines the project
status reporting and communication processes for task completion and budget.
Risk Level
4.1 Have communications been planned, identified and documented? Findings: Yes. A communication plan has been completed.
4.2 Is the Communications Plan being followed? Findings: Yes. The project team is consistently following the written
communications plan. All major communications are documented and are copied to the project SharePoint site. CAI will be
reviewing the Communications Plan and documents throughout the project to periodically to see if it is being followed and if it is
effective.
4.3 Does the project receive appropriate and timely executive and project sponsor attention? Findings: Yes. The Project Sponsor
and Project Team generally meet weekly to review key issues, risks and next steps/actions for the week. The Project Sponsor
is a member of the BMS Project Steering Committee.
4.4 Are project status and activities being monitored and reported in enough detail and with enough frequency to ensure early
detection of problems or schedule slippage? Findings: Yes. The project status, including the review of the project schedule, is
reported weekly at the BMS Project Team meetings. The team creates a weekly agenda they follow, and they have created a
register to record and track new or changes to Risks and Issues on a weekly basis. They have also instituted a project Kanban
board to track the backlog of activities, all blocking issues, and completed tasks. CAI will continue to attend weekly team
Kanban meetings to monitor issues and progress. During the months of December 2017 and January 2018, most of the project
effort was to complete the training on the AWARE system for the project SME’s.
Recommendation(s): None

5 QFP
2 5 - Risk and Issue Management
Project risks and issues are identified and appropriately managed. The Risk and Issue Management QFP provides an
assessment of the risk identification, mitigation strategy and contingency planning for high probability and/or high impact
risks. It also assesses the continuing validity of high impact assumptions.
Risk Level
5.1 Are project risks and issues identified and categorized as to likelihood and impact? Findings: Yes. The project maintains a
register to track and monitor risks using Priority / Probability / Impact as ways to prioritize workload. An additional issue register
is maintained and monitored by the project team. CAI is monitoring the team process for updating and mitigating all risks and
issues during the project.
5.2 Are appropriate risk and issue mitigation strategies in place with appropriate monitoring measures? Findings: Yes. A risk
register and mitigation plan are in place. The Project Team updates and discusses the risk register with the Project Sponsor
generally on a weekly basis and with the Steering Committee as necessary.
5.3 For high probability or high impact risks, are contingency plans developed in case the risk mitigation strategy fails? Findings:
Yes. CAI has observed that the project team is proactive in identifying risks and issues and immediately developing
contingency strategies when necessary. CAI continuously reviews any contingency plans for completeness and asks for
additional elaboration and monitoring should that be necessary.
5.4 Are ongoing risk identification, assessment and management processes in place and operating effectively? Findings: Yes. The
project teams meet weekly to review and assess the identified project risks. The project team is closely working with agency
Subject Matter Experts (SMEs) to ensure business requirements are being addressed in the project.
5.5 Have project assumptions been verified & appropriate monitoring measures been put in place to ensure failed assumptions do
not become risks? Findings: Yes. The project has identified the key assumptions as best as possible in the planning phase.
There were no surprises identified during the BEP initial implementation. Completion of the BEP GAP analysis and BEP
training went well during this last reporting period.

6 QFP 6 - BMS Vendor Performance

The BMS Vendor Performance. The BMS Vendor Performance QFP assesses the breadth and depth of the project’s vendor
performance.
Risk Level
6.1 Are Vendors involved and working with the BMS Project team on configuration, data conversion, and interface development?
Findings: Yes. The pace of vendor/project team interaction is intense. Both the vendor (Alliance) and BMS project team
members are in daily communication with project activities and status and completing the project deliverables according to the
project schedule. Discussions on data conversion are just beginning and CAI will be monitoring progress in this area.
6.2 Are the deliverables from the Vendor meeting overall performance criteria? Findings: Yes. Alliance is meeting all
performance expectations. CAI will be continually monitoring this QFP as the project progresses.
A new Alliance Project Manager has been assigned to the project in November, her name is Kellie Lamb. She is an
experienced Alliance PM and there was sufficient time to cross-train her with the departing PM (Jodi Hay) by January 19, 2018.
This is the “third” vendor PM on this project. CAI will be monitoring the effectiveness of this transition to the new alliance PM
during January and February 2018. Alliance has made assurances that the impact of the new PM will not impact the DSB BMS
project.
Recommendation(s): None.
7 QFP 7 - Technical Transition, Business Process Re-Engineering

The Technical Transition is adequately planned. This QFP assesses the preparation for managing and conducting the
technical transition.
Business Process Re-engineering is adequately planned. This QFP assesses the preparation for managing and
conducting the business transition.
(T) Technical Transition Risk Level
(B) Business Process Re-Engineering Risk Level
7.1 Has the Technical Transition Plan been defined? Findings: Yes. This is a COTS solution that is used in many other States.
The key technical issues were planning for the platform (Azure Government Cloud) and data security which has been
certified by WaTech as approved, dated October 9, 2017.

7.2 Has the Technical Transition Plan been reviewed and approved? Findings: Yes. CTS has approved the Azure Government
Cloud platform and the data security protocols for hosting the Alliance SaaS. (See above comment)
7.3 Has the Vendor architecture been assessed relative to the BMS architecture? Findings: N/A. This is a COTS system and
technical architecture is not an issue.
7.4 Are there appropriate resources to implement the Plan? Findings: Yes. Currently, it appears that the required resources
are available. The hiring of the Test lead and the Configuration Specialist were completed in November 2017 and the
onboarding and training and are progressing to meet the project schedule. From discussions with the Project Sponsor and
PM, the hiring of two temporary data entry staff is not necessary. A project team member has been able to assume that role
and is doing a great job in entering quality data.
7.5 Has the Business Process Re-Engineering Plan been defined? Findings: Yes. The project’s Business Analyst has done a
complete review of all business processes in the agency identifying and is documenting current-state business workflows and
is able to take that information not the adaptation phases of each module.
7.6 Has the Business Process Re-Engineering Plan been reviewed and approved? Findings: Yes. The Steering Committee
and Project Sponsor have approved the business re-engineering plans for the project.
7.7 Has the BMS workflow been assessed relative to changes from the System 7 “as-is” processes? Findings: Yes. The pre-
work by the project Business Analyst has identified the “as-is” or current workflows. Reviewing and documenting the current-
state work-flows is paying off as the project team and business SME’s are prepared and ready for the first business team to
move into adaptation (configuration) in the new AWARE system.
7.8 Are there appropriate resources to implement the Plan? Findings: Yes. The project has one highly skilled and capable
business analyst on the project team. It will be critical to ensure that this resource is available throughout the life of the
project.
8 QFP 8 - Project Organization and Leadership

The project is appropriately organized. The Project Organization and Leadership QFP assess the breadth and depth of
the project’s organization and the commitment to the project within the organization. This determines if the project’s
organizational structure can manage both tactical and strategic project issues.
Risk Level
8.1 The Steering Committee is comprised of executive decision-makers and is functioning? Findings: Yes. The BMS Steering
Committee is established with appropriate executive decision-makers and regularly twice per month.

8.2 Have Executive Sponsor(s) have been designated? Findings: Yes. The BMS Project Sponsor is the agency CIO, Mary
Craig.
8.3 Project Management roles and responsibilities with lines of authority and accountability have been defined, assigned and
agreed upon? Findings: Yes. The BMS Project Resource Management Plan v1.2 clearly defines the roles and
responsibilities, lines of authority and accountability. CAI will monitor if the Resource Management Plan needs to be updated
as the project progresses.
8.4 Is management committed to the project? Findings: Yes. Management has clearly communicated to the agency that they
are fully committed to the project and have allocated the required resources for the project to succeed.
8.5 Are management and Staff are committed to the project? Findings: Yes. One of the key success factors for this project is the
commitment by management and staff to move to the AWARE system. CAI has observed during this reporting period that
staff are fully engaged in the project at all levels.
8.6 Do Stakeholders Support and Buy-in to the project? Findings: Yes. The project is underway and staff buy-in to the project is
high. CAI will be monitoring stakeholder’s continued support as the project progresses. Stakeholder engagement is critical to
the project success.
8.7 Has a Change Leadership Plan been prepared? A project Change Leadership Plan identifies the steps and procedures that
will be followed should key project staff (e.g., Project Manager, Technical Lead, etc.) no longer be available. Findings: Yes.
Although the Project Charter does not refer to a change leadership plan, the project team has developed a Resource Plan
that provides the necessary steps and actions for project resourcing challenges.
8.8 Is the Change Leadership Process adequately supported by Agency Management? Findings: Yes. There is a solid
commitment by the DSB Steering Committee and the Project Sponsor to maintain and follow the Resource Plan.
8.9 Are there appropriate resources to implement the Change Leadership Plan? Findings: Yes. CAI will be reviewing the
adequacy of the Resource Plan as the project progresses.

9 QFP
7 9 - Project Resources
The project is appropriately resourced. The Project Resources QFP assesses three resource components: (1) The
capacity and skill set of the assigned project staff; (2) supporting tools and facilities; and (3) budget or financial resources.
Risk Level
9.1 Is the level of effort planned for each project deliverable at an appropriate activity level; and, is it reasonable? Findings: Yes.
A final project schedule has been approved by the Steering Committee the interim and major milestones in the project have
been baselined. Staffing for these activities is carefully monitored by the project team and also by the Alliance team. CAI has
observed a high degree of attention by the Project Team to ensure the business objectives are going to be met.
9.2 Are appropriate staff resources (skill set and quantity) available and assigned to complete the project? Findings: Yes. The
project team has been appropriately staffed with Subject Matter Experts (SMEs) identified across the agency and committed
to the project. The project team is monitoring, adjusting, and ensuring the right staff is available at the appropriate time in the
project schedule.
9.3 Are appropriate staff support resources (skill and quantity) available and assigned to provide on-going operations support?
Findings: Unknown currently. The current project plan discusses the cut-over and on-going implementation of the project
but only at a high-level. CAI will be reviewing these tasks and activities as the project progresses.
9.4 Are appropriate tools and other necessary facilities available and effectively utilized? Findings: Yes. The project team is
moving to a new location in the next several months that will allow the project team to be more centralized with larger rooms
for project staff.
9.5 Is the Budget (financial resources) sufficient to support the Project? Findings: Yes. The Executive Sponsor and PM are
monitoring the budget carefully and have found several planned and budgeted customizations that may not be necessary and
will save project costs and provide some flexibility in other project areas.
10 QFP 10 – Project/Quality Management and Reporting

The project is appropriately managed and quality controlled. The Project/Quality Management and Reporting QFP
assess the Project Management, Quality Assurance and Quality Control Processes and Plans in accordance with the
project methodology being used (e.g., waterfall, rolling wave or iterative, or Agile). Are the Plans in place and followed to
ensure project deliverables/releases meet requirements and are accomplished on time and within budget?
Risk Level

10.1 Have formal Project Management and Quality Management Plans been developed? Findings: Yes. The Project
Management Plan contains a section on quality management.
10.2 Are the Plans being followed? Findings: Yes. The project team is using appropriate PMI practices for managing the project.
10.3 Have appropriate metrics and processes been put in place to successfully manage the project? Findings: Yes. The Project
Manager is tracking tasks, tasks completion percentages, and vendor deliverables according to the project schedule. The
Project Manager is using the traditional PMI metrics for controlling and monitoring the project.
10.4 Have objective quality metrics been put in place for project deliverables? Findings: Yes. The project has acceptance criteria
for vendor deliverables.
10.5 Are Project Progress and Deliverables measured against the metrics? Findings: Yes. The working project schedule will be
used to monitor progress.
10.6 Are the results of the metric measurements reported to the appropriate sponsor, users, and other stakeholders? Findings:
Yes. The project team provides the major accomplishments and key issues at the monthly Steering Committee meetings.
10.7 Are appropriate corrective actions put in place when measurements are not acceptable? Findings: Yes.
10.8 Are appropriate status reports prepared for tracking and monitoring all project tasks? Findings: Yes. The Project Manager is
preparing, reporting and monitoring on the project schedule tasks and activities. As the project progresses, CAI will review
the current set of reports and processes for completeness and usability to reduce risk to the project.
11 QFP 11 - Budget Planning and Tracking

5
The project budget is appropriately planned, managed and tracked. The Budget Planning and Tracking QFP assess the
Project Budget Planning and Tracking Processes. Are the Plans in place and followed to ensure project deliverables meet
requirements and are accomplished on time and within budget?
Risk Level
11.1 Does the Project Manager and the Project Sponsor meet on a regular basis? Findings: Yes. The project sponsor is in daily
contact with the Project Manager to review and discuss any budgetary issues.
11.2 Is the Budget thoroughly planned and Budget to Actuals reported in a timely manner? Findings: Yes. CAI will be reviewing
the planned vs actual budget tracking during the next reporting period.

11.3 Are the appropriate funds budgeted in order to conduct required activities and complete and support the project? Findings:
Yes. The project team is drafting the document required by the OCIO’s office to receive implementation funds from the IT
Pool. The preliminary review of the document by OCIO staff found no red flags in the application and funding should be
approved in the next month.
11.4 Does the Project Manager maintain a tracking report of expenditure? Findings: Yes. CAI will be reviewing the project’s
expenditure tracking during the next reporting period.
12 QFP
6 12 - Scope and Change Control
The project scope is appropriately controlled. The Scope and Change Control QFP assesses the implementation and
adherence to change requests.
Risk Level
12.1 Is the scope is being adhered to? Note: Changes in scope usually impact the budget. Findings: Yes. The Project
Management Plan contains a section on scope management. The Project Sponsor and Project Manager follow this plan.
12.2 Are change requests appropriately identified, escalated, and resolved in a timely manner? Findings: Yes.
12.3 Are change requests effectively recognized, analyzed for impact, and approved prior to inclusion in the project scope?
Findings: Yes. The project team is adept at identifying all issues and impacts to the scope of the project. Their concerns are
discussed during daily standups and weekly team meetings and any keys issues are brought to the Steering Committee for
their review, discussion, and approval.
12.4 Is there a Change Management Plan that has been approved and adopted in the project? Findings: Yes.

13 5QFP 13 – Roles, Responsibilities, and Communications

The project is staffed with appropriate roles and responsibilities. Communications are effective. The Roles,
Responsibilities and Communications QFP examines the project status reporting and communication processes.
Risk Level
This QFP is similar to QFP #4 that focuses on Communications Planning.
13.1 Has a formal Communications Plan been developed? Findings: Yes. This is further expanded on in the OCM analysis to
ensure complete up and down communications in the project.
13.2 Is the Communications Plan being executed? Findings: Yes.
13.3 Are communications identified in the plan and implemented by the Project team effective? Findings: Yes.
The employee out-reach has been effective and excitement in the project is growing across the agency.
13.4 Are the external project communication dependencies included in project status reporting? Findings: Yes.
13.5 Are the project roles and responsibilities documented and understood by all parties? Findings: Yes. The Project Charter
clearly outlines roles and responsibilities for project staff.
14 QFP
6 14 - IT Architecture
The project adheres to architecture standards. The IT Architecture QFP verifies that the Project conforms to IT Architecture
standards.
Risk Level
14.1 The computing environment supports connectivity, portability, scalability, and interoperability? Findings: Yes
14.2 The Project supports the State of Washington IT Architecture Framework? Findings: Yes.
14.3 The Project supports the State of Washington IT Architecture Framework strategic objectives? Findings: Yes.
14.4 The Project supports the State of Washington IT Architecture Framework for increasing data security? Findings: Yes. The
project received approval from the Office of Cybersecurity on vendor security compliance.

15 QFP 15 - IT Acquisition Management

The project adheres to contractual standards. The IT Acquisition Management QFP examines that the vendor deliverables
meet the Project requirements and standards per the Contract Terms and Conditions.
Risk Level
15.1 Are procedures defined and documented to monitor and track vendor deliverables? Findings: Yes. The project has developed
acceptance criteria before approving payment on all deliverables. Deliverables are tracked against the newly approved project
schedule.
15.2 Are procedures defined and documented to measure vendor deliverables against project requirements and standards per the
contract terms and conditions? Findings: Yes. The PM uses the Alliance contract language to review and approve all
deliverables.
15.3 Do Project staff understand and follow documented procedures regarding vendor deliverables requirements? Findings: Yes.
The Project Manager is following all procedures for acceptance of Alliance deliverables.
16 QFP 16 - Project Library and Configuration Management

The project has an appropriate Project Library in place. The Project Library and Configuration Management QFP
examines if the Project has the appropriate document library to support Project Management Processes and a
Configuration Management Process in place to support System Delivery.
Risk Level
16.1 Has a Secure library has been defined? Findings: Yes. The project has a SharePoint library that contains most of the project
documentation. The project site is simple in design and navigating the site is easy.
16.2 Have procedures been documented for configuring and maintaining the library? Findings: Yes. The site will be updated and
maintained by two primary users, the Project Manager, and the project’s Business Analyst.
16.3 Are procedures documented for checking items in and out of the library Findings: Yes. The project site uses the standard
check-in and check-out procedures in the SharePoint system.
16.4 Are there contractor controls and monitoring in place? Findings: Yes.

16.5 Are there procedures for reviewing changes to items in the library? Findings: Yes. The project site uses the standard
features of SharePoint for changes to project documentation.
16.6 Is it relatively easy to access and find project documentation? Findings: Yes. It is important that the project team keep the
information on the SharePoint current.
16.7 Is there a process in place to archive older documents? Findings Yes. The project will be using the features of SharePoint
library to store and manage project documentation.
17 QFP 17 - System Definition Process

5
The system analysis and definition process are appropriate and thorough. The System Definition Process QFP assess
that the Project Requirements are supported in the Systems Design in accordance with the project methodology being
used (e.g., waterfall, rolling wave or iterative, or Agile).
Risk Level
17.1 Are the System and Business Requirements understood and confirmed? Findings: Yes. The project team has a completed
a full business analysis of the requirements in each of the business areas. The requirements will be matched against the
configurable items in the AWARE system through the adaptation process.
17.2 Are requirements traceable to design? Findings: Yes. (see the response in 17.1)
17.3 Are requirement change impacts understood, documented and incorporated into the Project Plan/Change Management
Plan? Findings: Yes. The vendor (Alliance) and the project team are fully aware of the change management impacts on the
project schedule and budget. The project team will match the business requirements to the system configurable items
through a GAP analysis. The first GAP analysis was completed in the Business Enterprise Program (BEP) area. Subsequent
GAP analyses were conducted on VR, Fiscal and OTC. CAI will be participating in the meeting to observe the process.
17.4 Are test conditions defined to validate requirements compliance? Findings: Unknown at this time. There is a process for
testing in the AWARE implementation. Since we are early in the implementation of the project CAI will be evaluating this
QFP question in subsequent monthly reports. From what CAI has observed during the project and the maturity of the
AWARE product and the professionalism of Alliance, we do not see any problems in the project.

18 QFP
5 18 - System Design Process
The system design process is appropriate and thorough. The Systems Design QFP assess the Systems Design are
reasonable and acceptable for implementation in accordance with the project methodology being used (e.g., waterfall,
rolling wave or iterative, or Agile).
Risk Level NOT RATED
18.1 Are specifications/designs in agreement with the system/business requirements? Findings: N/A, evaluation of this QFP will
occur in a future report as the implementation of the system becomes more mature.
18.2 Are the application specifications reasonable and acceptable? Findings: N/A, evaluation of this QFP will occur in a future
report as the implementation of the system becomes more mature.
18.3 IsI the system architecture reasonable and acceptable? Findings: N/A, evaluation of this QFP will occur in a future report as
the implementation of the system becomes more mature.
18.4 Are the Database Conversion and Migration specifications reasonable and acceptable? Findings: N/A, evaluation of this QFP
18.5 Are
A the Interface specifications reasonable and acceptable? Findings: N/A, evaluation of this QFP will occur in a future report
as the implementation of the system becomes more mature.
19 QFP 19 - Data Conversion/Migration

5
The data conversion and migration process is appropriate. The Data Conversion/Migration is well thought out for
migrating data to the new system in accordance with the project methodology being used (e.g., waterfall, rolling wave or
iterative, or Agile).
19.1 Are the Data Replication/Migration Strategy and Plans reasonable? Findings: N/A, evaluation of this QFP will occur in a
future report as the implementation of the system becomes more mature.
19.2 Do the plans include data cleanup, testing, and user acceptance criteria? Findings: N/A, evaluation of this QFP will occur in
a future report as the implementation of the system becomes more mature.

19.3 Is the Data Replication/Migration complete? Findings: N/A, evaluation of this QFP will occur in a future report as the
implementation of the system becomes more mature.
19.4 Has the replicated/migrated data been tested to verify they function according to the specifications? Findings: N/A,
evaluation of this QFP will occur in a future report as the implementation of the system becomes more mature.
19.5 Are the unit test scripts complete and thorough with respect to the business processes? Findings: N/A, evaluation of this
QFP will occur in a future report as the implementation of the system becomes more mature.
20 QFP 20 – Configuration/Construction Process

5
The configuration and development of the new system are appropriate. The Configuration/Construction Process QFP
examines the appropriateness of project’s processes to meet requirements in accordance with the project methodology
being used (e.g., waterfall, rolling wave or iterative, or Agile).
20.1 Have the configuration specifications been verified to meet requirements? Findings: N/A, evaluation of this QFP will occur in
20.2 Is the Development Methodology appropriate and is it being followed? Findings: N/A, evaluation of this QFP will occur in a
20.3 Have the Test Plans been updated to reflect the configuration specifications? Findings: N/A, evaluation of this QFP will
20.4 Have the specific system modules been tested to verify they function according to the specifications? Findings: N/A,
20.5 Has the converted/migrated data been tested to verify they function according to the specifications? Findings: N/A,
20.6 Are the unit test scripts being complete and thorough with respect to the business processes? Findings: N/A, evaluation of
this QFP will occur in a future report as the implementation of the system becomes more mature.

21 5QFP 21 - Testing (Functional, Capacity, and Performance)

The project is appropriately tested. The Testing (Functional, Capacity and Performance) QFP assesses the acceptance
testing processes and plans are in place and meet the operational needs of the system and verify and validate
acceptable compliance to requirements Design in accordance with the project methodology being used (e.g., waterfall,
rolling wave or iterative, or Agile).
21.1 Has a formal Test Management Plan been developed? Findings: N/A, evaluation of this QFP will occur in a future report as
21.2 Is the Test Plan being followed? Findings: N/A, evaluation of this QFP will occur in a future report as the implementation of
the system becomes more mature.
21.3 Do the documented functional specifications meet the business needs? Findings: N/A, evaluation of this QFP will occur in
21.4 Are the business users involved in establishing the functional acceptance testing scope and standards? Findings: N/A,
21.5 Are the functional acceptance test processes appropriate and are results monitored and tracked? Findings: N/A, evaluation
of this QFP will occur in a future report as the implementation of the system becomes more mature.
21.6 Do the capacity and performance specifications match operational needs? Findings: N/A, evaluation of this QFP will occur
in a future report as the implementation of the system becomes more mature.
21.7 Are the capacity and performance acceptance test processes appropriate and are results monitored and tracked? Findings:
N/A, evaluation of this QFP will occur in a future report as the implementation of the system becomes more mature.
21.8 Is comprehensive end-to-end functional, capacity and performance acceptance testing planned and performed for all
software, hardware, and telecommunication components? Findings: N/A, evaluation of this QFP will occur in a future report
21.9 Are infrastructure conditions (down to the computing hardware level) that may affect the application being considered,
tested and resolved? Findings: N/A, evaluation of this QFP will occur in a future report as the implementation of the system
becomes more mature.
21.10 Was a defect log maintained and effective corrective actions taken? Findings: N/A, evaluation of this QFP will occur in a

22 QFP 22 - User Acceptance, Business Process Transition

5
The User Acceptance Process is defined and appropriate. The User Acceptance, Business Process Transition QFP
examines the methods, test plans, test procedures, and test results required to ensure the delivered system meets
customer requirements.
22.1 Has a User Acceptance Test (UAT) Plan been prepared? Findings: N/A, evaluation of this QFP will occur in a future report
22.2 Are the acceptance test scripts complete? Findings: N/A, evaluation of this QFP will occur in a future report as the
T
22.3 Are testing standards understood and followed? Findings: N/A, evaluation of this QFP will occur in a future report as the
22.4 Is a defect log maintained and are corrective actions effective? Findings: N/A, evaluation of this QFP will occur in a future
22.5 Have the Business Process Changes been effectively tested? Findings: N/A, evaluation of this QFP will occur in a future
22.6 Have user acceptance criteria been met? Findings: N/A, evaluation of this QFP will occur in a future report as the
23 QFP 23 – Training
5
The project staff and system users are appropriately trained in a timely manner. The Training QFP assesses the
training plans and materials and methods for training are appropriate and complete.
Risk Level
23.1 Has formal Training Plan been developed? Findings: Yes. Alliance has a complete training plan for their product line. The
project team has been proactive in getting staff trained on the appropriate AWARE modules before adaptation sessions
begin to ensure understanding of how the module works and is appropriately configured to meet the business requirements.
23.2 Are the Training Plans being followed? Findings: Yes.

24 QFP 24 - Inter/Intra-Agency Coordination and Implementation

5
Inter-Agency dependencies are in place. The Inter/Intra-Agency Coordination and Implementation QFP examines the
policies, SLAs, and communications are in place to assure dependencies are understood and agree too.
Risk Level
24.1 Is there an interagency agreement, including the scope of work, resources, and schedule? Findings: Yes.
24.2 Is the Communication Plan between BMS and the other agency stakeholders defined and being followed? Findings: Yes.
24.3 Are issues reported and resolved according to the Issue Management Plan? Findings: Yes.
24.4 Are
2 system interface requirements completed and approved, including acceptance criteria? Findings: Yes.
24.5 Are
2 the Business Processes (workflows and procedures) completed and approved? Findings: Yes.
24.6 Was
2 any required training completed and was it acceptable? Findings: Yes. CAI observed VR training in the Seattle Office
in January. The training was not as effective as it could have been. Both CAI and the project team made suggestions to the
Alliance Team and they began to make the necessary adjustments immediately. CAI is favorably impressed with the rapid
response to improving DSB training needs.
24.7 Have the implementation acceptance criteria been met and approved? Findings: Yes.
2
25 QFP 25 - Implementation Process

5
The new system has been successfully moved into the production environment. The Implementation Process QFP
assesses the plans for implantation are define, including all documentation to assure knowledge transfer.
25.1 Is the installation specification complete and reasonable? Findings: N/A, evaluation of this QFP will occur in a future report
25.2 Are the training plans and the new Business Processes (workflows and procedures) complete and approved? Findings: N/A,
2

25.3 Was the actual training conducted acceptable by the staff? Findings: N/A, evaluation of this QFP will occur in a future report
2
25.4 Was system documentation completed and approved? Findings: N/A, evaluation of this QFP will occur in a future report as
2
25.5 Have the implementation acceptance criteria been met and approved? Findings: N/A, evaluation of this QFP will occur in a
2
26 QFP 26 - Deployment Process

5
The new system has been successfully deployed. The Deployment Process QFP examine the deployment was
successful and all acceptance criteria were met.
26.1 Are deployment plans complete, reasonable and approved? Findings: N/A, evaluation of this QFP will occur in a future
26.2 Was training the BMS staff completed and acceptable? Findings: N/A, evaluation of this QFP will occur in a future report as
26.3 Is user documentation complete and approved? Findings: N/A, evaluation of this QFP will occur in a future report as the
26.4 Have deployment acceptance criteria been approved? Findings: N/A, evaluation of this QFP will occur in a future report as

27 QFP 27 – Interface Design and Implementation

5
The system interfaces have been successfully designed and Implemented. The Interface Design and Implementation
QFP examine whether the system interfaces were designed and implemented successfully and whether acceptance
criteria were met.
27.1 Is the Interface design plan complete, sufficiently detailed, reasonable and approved? Findings: N/A, evaluation of this QFP
27.2 Have the owners of both sides of the interface been engaged and agree with the interface design plan? Findings: N/A,
27.3 Is the design progressing per the plan? Findings: N/A, evaluation of this QFP will occur in a future report as the
27.4 Are encountered problems and issues being identified and resolved in a timely manner? Findings: N/A, evaluation of this
QFP will occur in a future report as the implementation of the system becomes more mature.
27.5 Is interface testing progressing in a timely manner and with expected results? Findings: N/A, evaluation of this QFP will
27.6 Are the interfaces documented sufficiently including ongoing support and maintenance procedures? Findings: N/A,

Appendix A – Project Activities, Meetings, Deliverables

Meetings Attended:
The following project meetings were attended in-person or remotely for this reporting period.
Meeting Name Date(s)
Project Team Meetings 12/20, 1/31
BMS Steering Committee meetings 1/25
AWARE OTC training in Seattle 1/16
OCIO briefing with the new OCIO consultant (Whitney Dickinson) 1/12
Meeting with the BMS PM to discuss project status 1/23
Documents Reviewed:
The following project documents were reviewed for content and purpose during this reporting period.
Document Name Comments
Incorporated comments if
Reviewed DSB comments on the QA report
appropriate
Weekly progress report summaries As they are produced
Needs to be updated as
BMS Risk Register
changes are being made
BMS Issue Register
BMS Decision Log
BMS Project Action Log Is this document still used?
OCIO December Project Status Report
Working Integrated Project Schedule 20171226 (Note: This
project schedule was approved by the BMS Steering Committee
with the recommended date changes identified by the BSM PM)
AWARE Infrastructure Assessment
Reviewed the SharePoint Site Library for project artifacts
Office of CyberSecurity - Design Review Summary - DSB BMS
Aware Data Conversion Overview
BMS Communications Plan r1
BMS OCM Roadmap
Data Conversion Build Tracking Matrix - WA DSB Draft
DSB BMS OCM Plan v1
WA DSB Data Conversion Plan
WA DSB High-Level Test Management Plan
Prepared by CASE associates Inc. 32 Contract # 430000266

Documents / Reports Delivered:
The following project documents were delivered during this reporting period.
Document Name
DSB BMS January 2018 Monthly QA Report #5
OCIO Project Website posting of the December/January QA Report
Prepared and updated DSB SharePoint Matrix and shared with OCIO consultants to indicate
where key project documents are located on the BMS SharePoint site.
Documents / Reports / Meetings for Next Month:
The following project documents will be reviewed/delivered and meeting attended in next month’s
reporting period.
Document Name
DSB BMS 2017 February Monthly QA Report
Post QA report to the OCIO project website
BMS Risk Register

BMS Issue Register
BMS Decision Log
BMS Integrated Project Schedule (approved 1/25/2018)
Data Conversion Plan
Review the SharePoint Site for project artifacts
Review Alliance deliverables for the month
AWARE adaptation sessions and results
Attend Project Team meetings
Review the VR Adaptation review and adoption process
Review the IT Funding Pool documentation for funding the implementation phase of the project
Review the Test Case Management process for BEP and VR if available next month
Attend and present QA findings at Steering Committee meetings
Attend Data Mapping sessionjs in mid-February
Review the AWARE Analytics process for the project
Review the data extract process for the project
Review process for BEP data entry quality

Appendix B - QA Methodology and Report Structure

CAI started its QA effort for the Business Management System Project in June 2017. This BMS QA
Report is delivered monthly as specified in the Quality Assurance Management Plan (QAMP).
Purpose
The purpose of this report is to provide DSB with an independent, in-depth (deep dive) quality
assessment of the Business Management System Replacement Project. CAI assesses the overall
health of the project, as well as how key project stakeholders feel about the effort and new system. CAI
measures progress, or lack thereof, of key project deliverables and provides DSB with
recommendations on how to address identified issues.
Methodology
CAI Consultants used the following methodology to complete this quarterly assessment and to prepare
this report:
1. Review project documents and deliverables. The documents that are reviewed are listed in
Appendix A under the Documents Reviewed section of this report.
2. Attend meetings, listen and participate in daily/weekly conference calls with project team
members and stakeholders. Conduct interviews with BMS Project Manager, project team
members, project participants and other stakeholders to determine the project's status and
identify possible issues and risks. CAI activities during the reporting period are listed in
Appendix A.
3. Based on CAI's informed judgment, the documents reviewed, meetings attended, and
interviews conducted, CAI comes to an independent, unbiased opinion of the status and the
health of the project.
CAI's knowledge and experience are based on the following:

1. Project Management Institute, "Project Management Body of Knowledge" (PMBOK). We guide
and mentor our clients in applying project management and quality assurance methodologies.
2. Total Quality Management concepts and the Institute of Internal Auditors Process Audit
Standards.
3. The Control Objectives for Information and Related Technology (CobiT) standards developed by
the Information Systems Audit and Control Foundation for use in Information Technology Audits.
4. The tenets of software management, including the functions of Quality Assurance (QA) (Per
IEEE-Std 730) and Independent Verification and Validation (IV&V), that is found in Managing
the Software Process by Watts Humphrey.

Definition of Risk Factors:
NR Not Reviewed This Reporting Period
The Quality Focal Point is impacting the project and needs immediate
attention. CAI includes a Recommendation for every QFP rated Alert. The
Alert recommended action is a corrective measure to improve the QFP that is
currently impacting the project.
The Quality Focal Point needs some improvement, so it won’t impact the
project. CAI includes a Recommendation for every QFP rated Attention.
Attention
The recommended action is a corrective measure to improve the QFP, so it
won’t impact the project.
The Quality Focal Point is stable and not currently impacting the project.
CAI may include a Suggestion in a QFP rated as Stable. CAI rated the
Stable
QFP as stable because it is not impacting the project at this time. The
suggested action is a preventive measure to keep the QFP stable.
Definition of Next Month Indicators:

The next month indicators signify expected changes in the QFP ratings.
- This Quality Focal Point (or group of QFPs) is expected to have the same rating in
next month’s Periodic QA Evaluation.
- The risk is increasing the Quality Focal Point. It may be changed to Alert in
next month’s evaluation
Attention or Stable in next month’s evaluation.
- The risk is increasing the Quality Focal Point. It may be changed to Attention
or Alert in next month’s evaluation.
- The risk is decreasing on the Quality Focal Point. It may be changed to Stable in
next month’s evaluation.

Appendix C – QA Recommendations and Response Tracking
Open QA Recommendations
The following table lists the open QA Recommendations from the baseline and previous Assessments, along with their status and any revisions. As a note, new recommendations are
identified above and not included in this table:
There are no open recommendations in the January 2018 monthly QA report #5.
ID BMS Action CAI
Recommendation QFP / QA Rating /
Impact Owner Taken and Confirmation of
Date – Source Description Recommendation Status
Status Status
Closed QA Recommendations and Agency Responses
These are the Closed QA Recommendations from previous Assessments.
ID Recommendation Date – Rating / BMS Action Taken and CAI Confirmation of

QFP / Description QA Recommendation Impact Owner
Source Status Status Status
#2 Monthly Report August QFP 3 – Project CAI recommends the Project High High Project During contract Confirmed Completion
2017 Schedule Manager prepare a detailed Manager negotiation, the vendor Date:
project schedule during the determined that the
(Continued contract negotiations to detailed project schedule 1/25/2018
recommendation for the ensure the deliverables, would be developed by
September, October, and deadlines, and staffing the end of January 2018. The final project plan
November monthly resources are aligned with was approved at the
reports) the contract language. As of 12/20/2017, DSB January 25, 2018, BMS
and Alliance have Steering Committee
completed the detailed meeting.
schedule for the BEP
Implementation. It is
located on SharePoint
as part of the “Working

Integrated Project
Schedule”.
DSB is working closely

with Alliance to
elaborate the remaining
VR schedule by 1/31/18.
Target completion:
January 31, 2018
#1 Readiness Assessment CAI recommends finalizing Medium High Project We are currently working Confirmed Completion
June 2017 the staffing plan and Sponsor with agency executives Date:
determine how to best backfill to firm up the project 11/17/2017
current operations for key Staffing Plan and create
staff that will be working on the best processes for
the BMS Replacement backfill. The final The September 29,
Project. Ensure that Staffing Plan will be 2017, target date is
Executive Management dependent on the joint reasonable. CAI will be
supports the need for a clear resource plan developed reviewing the status of
plan that the Agency staff feel with the vendor during this recommendation in
will be sufficient. contract negotiations. the next monthly report.
Target completion:
September 29, 2017
#2 Readiness Assessment CAI recommends providing High High Project To date, there have Confirmed Completion
June 2017 more rigor around the Manager been two formal Date:
execution of the Project communications 11/17/2017
Communication Plan. Ensure distributed to all staff
users, stakeholders and from executive
project participants are kept management. These The September 15,
apprised of the status of the communications 2017, target date is
Project, even if it means provided a personable, reasonable. CAI will be
explicitly stating delays or the detailed overview of the reviewing the status of
reason(s) for the apparent project, what to expect, this recommendation in
need for minimal what resources are the next monthly report.
communication. (It should be needed internally and
ongoing status regarding

noted that CAI believes that project funding.
Project communications are Legislative funding
an area the Project has approval was received
already noted has been the end of June 2017.
improving and feels this risk Project communications
is low.) are tracking to schedule.
Stakeholder analysis by
the OCM lead has
engaged over twenty
stakeholders statewide
in discovery phase for
the project. In addition
to organizational
attributes, readiness,
and initial OCM risk
analysis, all
organizational and
stakeholder data will be
used to develop
comprehensive, robust
change plans for
communications,
engagement, and
training specific to the
agency by August 2017.
These activities align
with the project
management plan and
scheduled tasks for
OCM.
Target completion:
September 15, 201
#3 Readiness Assessment CAI recommends that the High High Project The Project Confirmed Completion
June 2017 content of the Project Manager Management Plan Date:
Management Plans must (PMP) includes an Org 11/17/2017
be clear on how the Chart and
project will be managed, Roles/Responsibilities of
specifically: each role in the The September 29,
decision-making 2017, target date is

a. Update the Governance process. The Decisions reasonable. CAI will be
Model for the project - and Dispute Resolution reviewing the status of
specifically, the decisions section of the PMP will this recommendation in
are made (authority) and be updated to include the next monthly report.
memorialized in the more information on how
project. decisions are made. It
b. For the upcoming phase, includes the voting
the project should refine process of the Steering
the estimated effort for Committee and how
each of the deliverables decisions will be logged
and re-validate the and tracked on the
schedule and resource project. Attachment A is
capacity. If needed, take the most current version
appropriate action by: of the Project
Management Plan. b.
I. Adjusting the schedule
DSB will adjust the PMP
ii. Reducing scope to meet for actual deliverables,
the schedule schedule, and resource
iii. Adding resource capacity capacity after contract
c. Ensure the control negotiations with the
system is appropriate and vendor. c. DSB will
capable of monitoring the adjust the PMP for
schedule and budget. monitoring and control
Standards are defined standards after contract
and agreed to for negotiations with the
comparing planned to vendor. d. DSB will
actual performance. adjust the PMP for
estimated time and cost
d. The project is capable of
after contract
estimating time and cost
negotiations with the
to completion of each
vendor.
project milestone.
Target completion:
September 29, 2017
#4 Readiness Assessment CAI recommends beginning High High Project At the time of QA Completed June 30,
June 2017 to identify alternative streams Sponsor Readiness Assessment, 2017
of funding if the project the state budget had not
budget is not approved by the been signed. On June NOTE: DSB is still in the
legislators. 30, 2017, the final state process of completing
budget was signed and the Investment Plan but

DSB received funding this requires a contract
for the project. with the sole source
Alternative streams of vendor. Negotiations
funding will not be are close to completion.
needed. The BMS
Project made it into the
Senate IT Pool with only
two other projects.
Completed: June 30,

2017
#1 Monthly Report August QFP 8 - Project CAI recommends that the Medium Medium Project Human Resource Confirmed Date:
2017 Organization and project create a Change Sponsor Management Plan is 11/17/2017
Leadership Leadership Plan and get being developed. It has
approval from the BMS been reviewed with key
Steering Committee. internal stakeholders CAI confirmed that the
and will gain final Steering Committee
approval by the BMS approved the project
Steering Committee in resource management
November. plan.
Target completion:
November 10, 2017

DSB BMS 201801 January Monthly QA Report #5 v1.1

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

DSB BMS 201801 January Monthly QA Report #5 v1.1

Transféré par

Droits d'auteur :

Formats disponibles

Washington State

Department of Services for the Blind

Business Management System (BMS)

QA Monthly Report #5 – January 2018

Dan Kruger, Project Lead, Quality Assurance Consultant

Document Revision History

Version Date Comments

Prepared by CASE Associates Inc. Page 2 Contract # 430000266

Prepared by CASE Associates Inc. Page 3 Contract # 430000266

QFP 27 – Interface Design and Implementation................................................................... 31

Prepared by CASE Associates Inc. Page 4 Contract # 430000266

Overall Project Progress

Major Risks Identified During the Current Monthly Assessment Process

Prepared by CASE Associates Inc. Page 5 Contract # 430000266

Overall Project Health

Health Factor Rating Comments

Overall Project QFP Status – Stable

QFPs rated as Stable

Prepared by CASE Associates Inc. Page 6 Contract # 430000266

 QFP 16: Project Library and Configuration Management

QFPs rated as Needs Attention

QFPs rated as needing Immediate Attention

QFPs Not Rated – NR

Quality Focal Point (QFP) Summary Chart

Monthly Quality Focal Point Assessment (QFP) Assessment

Prepared by CASE Associates Inc. Page 7 Contract # 430000266

Monthly Quality Focal Point Assessment (QFP) Assessment

Prepared by CASE Associates Inc. Page 8 Contract # 430000266

Monthly Quality Focal Point Assessment (QFP) Assessment

Prepared by CASE Associates Inc. Page 9 Contract # 430000266

Recommendations for QFPs Rated Stable (Stable)

Recommendations for QFPs Rated Needs Attention

Recommendations for QFPs Rated Alert (Alert)

Prepared by CASE Associates Inc. Page 10 Contract # 430000266

Detail Quality Focal Point Assessment

Prepared by CASE Associates Inc. Page 11 Contract # 430000266

Prepared by CASE Associates Inc. Page 12 Contract # 430000266

Prepared by CASE Associates Inc. Page 13 Contract # 430000266

Prepared by CASE Associates Inc. Page 14 Contract # 430000266

Prepared by CASE Associates Inc. Page 15 Contract # 430000266

6 QFP 6 - BMS Vendor Performance

7 QFP 7 - Technical Transition, Business Process Re-Engineering

Prepared by CASE Associates Inc. Page 16 Contract # 430000266

8 QFP 8 - Project Organization and Leadership

Prepared by CASE Associates Inc. Page 17 Contract # 430000266

Prepared by CASE Associates Inc. Page 18 Contract # 430000266

10 QFP 10 – Project/Quality Management and Reporting

Prepared by CASE Associates Inc. Page 19 Contract # 430000266

11 QFP 11 - Budget Planning and Tracking

Prepared by CASE Associates Inc. Page 20 Contract # 430000266

Prepared by CASE Associates Inc. Page 21 Contract # 430000266

13 5QFP 13 – Roles, Responsibilities, and Communications

Prepared by CASE Associates Inc. Page 22 Contract # 430000266

15 QFP 15 - IT Acquisition Management

16 QFP 16 - Project Library and Configuration Management

Prepared by CASE Associates Inc. Page 23 Contract # 430000266

17 QFP 17 - System Definition Process

Prepared by CASE Associates Inc. Page 24 Contract # 430000266

19 QFP 19 - Data Conversion/Migration

Prepared by CASE Associates Inc. Page 25 Contract # 430000266

20 QFP 20 – Configuration/Construction Process

Prepared by CASE Associates Inc. Page 26 Contract # 430000266

21 5QFP 21 - Testing (Functional, Capacity, and Performance)