Software Quality Operational Readiness Review (ORR) Checklist

Date(s) of Assessment: ____________ Project: ______________________________________

Assessor(s): _____________________ Review Examined: _____________________________
_______________________________________________________________________________

Y, N, F, O Comments
NA
REVIEW PREPARATION
1 Have standards been identified to
1 clearly define the review process?
2 Were guidelines used to prepare for the
2 review?
Has the project submitted any request
3
for deviations or waivers to the defined
3
process?
4 Have entrance and exit criteria been
4 established for the review?
5 Was an agenda prepared and distributed
5 in advance of the review?
6 Was the review package provided with
6 ample time to review?
7 Were the appropriate stakeholders in
7 attendance?
REVIEW CONTENT
8 Were the goals of the review and any
8 review prerequisites provided?
9 Was the review process addressed,
9 including the method for capturing
Requests for Action (RFAs), risks, or
issues?
1 Was status given on action items from
0 the Acceptance Review?
1

1 Was status included on action items

1 from all previous software reviews?

Revision: 2.0 Page 1 of 6

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist
Y, N, F, O Comments
NA
1 Are the pertinent stakeholders involved
2 in approval of the review material?
1

1 Was an overview of the software

3 project/system provided (e.g., mission
1 goals, key functionality, operational
characteristics)?
1 Was a review/status of top-level
4 requirements and derived requirements
1 provided?

1 Was the relationship of requirements to

5 support provided (i.e., typical, critical,
special or contingency)?
1 Were organizational interfaces and
6 responsibilities defined?

1 Were operational support scenarios

7 identified?

1 Were the operational facilities (e.g.,

8 hardware environment, network) and
their availability identified?
1 Was status provided for all software
9 components?

2 Were test summary reports and overall

0 performance results presented?
1

2 Have all requirements been verified?

1

2 Were any workarounds and non-

2 functioning software components
1 identified and documented?

2 Was the operational problem escalation

3 process presented?
2

Revision: 2.0 Page 2 of 6

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist
Y, N, F, O Comments
NA
2 Was the operational emergency
4 notification process identified including
2 telephone contact numbers?

2 Is there evidence that all software

5 operations personnel are appropriately
2 trained?

2 Were security issues presented?

6

2 Were safety issues presented?

7
2

2 Are there any Configuration

8 Management (CM) issues identified
2 concerning the software operations?

2 Is there evidence that the software is

9 under CM control and change
2 procedures are being implemented as
specified in the CM Plan?
3 Is there evidence that all software
0 procedures and associated
documentation is under CM control and
change procedures are being
implemented as specified in the CM
Plan?
3 Is there evidence that software quality
1 procedures are being implemented as
2 specified in the SQ Plan?

3 Has SQ completed all audits and

2 reviews applicable to the software and
2 the affected system?

3 Are technical risks, mitigation plans,

3 and issues documented with plans for
2 tracking and closure?

Revision: 2.0 Page 3 of 6

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist
Y, N, F, O Comments
NA
SOFTWARE DOCUMENTATION STATUS
3 Does the review package address the
4 status of software documentation as
2 specified in the project plan?

3 Are operational documents and

5 contingency plans addressed in the
3 review package?

3 Was status provided for all system

6 documentation (e.g., OPS and
Maintenance Plans)?
3 Was status provided for all interface
7 documents with external organizations?

POST REVIEW ACTIVITIES

3 At the conclusion of the review is an
8 understanding reached on the validity
3 and degree of completeness of the
Operational Readiness Review?
3 Did all designated parties concur in the
9 acceptability of the Operational
3 Readiness Review?

4 Are there any risks, issues, or request

0 for actions (RFAs) that require follow-
3 up?

4 Is there a process in place for reviewing

1 and tracking the closure of risks, issues,
3 or RFAs?

4 Have all artifacts been placed under

2 formal configuration control (e.g.,
3 review packages)?

4 Were Lessons Learned addressed and

3 captured?
3

REFERENCE ITEMS/DOCUMENTS

Revision: 2.0 Page 4 of 6

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist
Y, N, F, O Comments
NA
IEEE Standard for Software Reviews, IEEE STD 1028-1997
Managers’ Handbook for SW Development, SEL-84-101

Review Definition – The ORR examines the actual system characteristics and the procedures used
in its operation and ensures that all flight and ground hardware, software, personnel, procedures,
and user documentation accurately reflect the deployed state of the system.

Revision: 2.0 Page 5 of 6

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist

Date(s) of Assessment: ______________ Project: _______________________________________

Assessor(s): ______________________ Review Examined: ______________________________
__________________________________________________________________________________

COMMENTS PAGE _____ of _____

Comments from assessment

Revision: 2.0 Page 6 of 6

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.