DO-178B and Safety-Critical Software

Technical Overview
Joseph Wlad Product Marketing Manager Wind River Alameda, CA

7/23/01

2000 Wind River Systems, Inc.

Agenda
n

DO-178B Overview
Background and History Certification Levels

Software Verification
Software Safety and Level A and Level B

VxWorks Real-Time Operating System and DO-178B

Certifiable Subset Overview

7/23/01

2001 Wind River Systems, Inc.

DO-178B Overview
n

DO-178B: Software Considerations in Airborne Systems and Equipment Certification, circa 1992
Evolved from DO-178A, circa 1985

DO-178B is a guidance document only and focuses on software processes and objectives to comply with these processes
Developed by RTCA, Inc (a not for profit company) and its members to ensure that software meets airworthiness requirements

Called out in many certification requirements documents as the recommended method to obtain approval of airborne software
Design Approvals through FAA Technical Standard Orders and Supplemental Type Certificates, among others Others calling out DO-178B: Military programs, Nuclear, Medical

Many other standards exists: SEI-CMM, DEF STAN 00-55, ISO, DOD-2167, IEC 61508
2001 Wind River Systems, Inc. 3

7/23/01

Working Groups and Committees create and evolve RTCA DO-178B/ED-12B

RTCA SC-167
8B O-17 D B D-12 E

EUROCAE WG-12

CAST (Certification Authority Software Team)

Avionics Industry

Cast Position Papers

SC-190 //WG-52 SC-190 WG-52

RTCA DO-248,others

7/23/01

2001 Wind River Systems, Inc.

DO-178B Overview
n

DO-178B is not prescriptive

Vendors are allowed to decide how objectives are satisfied

DO-178B objectives vary, depending upon how software failures can affect system safety Consider two aircraft examples
1) Software controlling the coffeemakers in the aft galley fails
Outcome: Likely some grumpy customers, but passenger safety not compromised (air rage issues due to lack of coffee aside)

2) Software controlling the aircraft during an automatic landing in zero visibility conditions fails
Outcome: Possibly catastrophic and lives lost
n

Obviously these two software applications need not be developed to the same rigor

7/23/01

2001 Wind River Systems, Inc.

DO-178B Overview
n n

For this reason, DO-178B defines five software levels Each level is defined by the failure condition that can result from anomalous software behavior Failure Condition Failure Condition Catastrophic Hazardous/Severe - Major Major Minor No Effect Software Level Software Level Level A Level B Level C Level D Level E
2001 Wind River Systems, Inc. 6

7/23/01

DO-178B Overview
n

Once a system safety assessment is done and the safety impact of software on is known then the level is defined Level A has 66 objectives, Level B 65 objectives, Level C 57 objectives, Level D 28 objectives Does DO-178B help make software safe?
Maybe: Heuristically, it appears to help but absence of failures is not a guarantee that the process helped eliminate them

How do we know when software is safe?

7/23/01

2001 Wind River Systems, Inc.

DO-178B Overview
We Dont Know !! We Dont Know !!

What is our best guess about software safety?

When applicable processes have been followed When the code has been verified from within When this has been checked and checked... and checked.. and checked... and checked.

7/23/01

2001 Wind River Systems, Inc.

DO-178B Overview
n

But, use of standard processes and compliance with predetermined objectives help avoid the common pitfalls of software development DO-178B defines the following processes (as well as objectives for each):
Planning Process Development Process Requirements Process Design Process Coding and Integration Process Testing and Verification Process Configuration Management Process Quality Assurance Process
2001 Wind River Systems, Inc. 9

7/23/01

DO-178B Overview
n n

Each process has inputs, outputs and transition criteria Descriptions of evidence needed to demonstrate an objective has been satisfied is included
For example: Is the source code verifiable?
Are analyses or tests provided that show the source code does not contain structures that can not be tested

The important point is that all these software lifecycle processes are linked in any given application: the lifecycle activities must be traceable!

7/23/01

2001 Wind River Systems, Inc.

10

Traceability
Review Review Review Review Review Test Results Test Procedures

Source Code Design

Requirements

Linkage
7/23/01 2001 Wind River Systems, Inc. 11

Software Verification

7/23/01

2001 Wind River Systems, Inc.

12

Software Verification
DO-178B Definition: Verification is not simply testing. Testing, in general, cannot show the absence of errors. As a result the DO-178B subsections use the term verify instead of test when the software verification process objectives being discusses are typically a combination of reviews, analyses and tests.

7/23/01

2001 Wind River Systems, Inc.

13

Software Testing
Black Box Testing Requirements Based Testing

White Box Testing Decision Coverage Boundary Value

DO-178B discusses requirements-based testing and coverage analysis

7/23/01 2001 Wind River Systems, Inc. 14

Level A and Software Safety

n

Level A Software whose anomalous behavior, as shown by the system safety assessment process, would cause or contribute to a failure of system function resulting in a catastrophic failure condition for the aircraft.

Software Safety - Ensure & verify that software takes Positive Measures to enhance the safety of the system & control errors that reduce the safety of the system. Added Benefits include: Higher reliability Improved maintainability More robust system
Level A requires that compiler added functionality be addressed
If compiler adds range checking, divide by zero, etc, then applicant must test these features

7/23/01

2001 Wind River Systems, Inc.

15

Differences between Level A and Level B

n

Independent verification of
Software Design process Source Code compliance Source Code accuracy Object Code robustness Test Objectives

n n n

Test coverage (Modified Condition/Decision) optional for Level B Level A: MCDC Testing Level B: Decision Coverage Level C: Statement Coverage

Analysis rigor much more severe for level A

7/23/01 2001 Wind River Systems, Inc. 16

Multiple Condition/Decision testing

if A=0 and B<2 and C>5 then P; ...... A=0 T F T T B<2 T T F T C>5 T T T F P T F F F

For Level A, each outcome must be tested once

7/23/01 2001 Wind River Systems, Inc. 17

VxWorks and DO-178B Certification

7/23/01

2001 Wind River Systems, Inc.

18

Software Components of a System

System Target System User Code VxWorks Operating System

7/23/01

2001 Wind River Systems, Inc.

19

VxWorks Description
n n

Commercial RTOS enviroment in use for > 10 years VxWorks consists of:
High performance Real-time Kernel I/O System: Network, serial, pipes, drivers, etc. Utility Libraries: timers, interrupts, messages, memory allocation, etc. Shared memory objects for multiple processors Board Support Packages: drivers, timers, memory mapping, etc.
Over 100 targets supported

Tools: simulator support, logic analyzer and performance evaluation SLOC: 2,000,000 lines
BSPs and drivers: 800,000 lines Network: 250,000 lines

architectures supported: 32+

7/23/01

2001 Wind River Systems, Inc.

20

Real-time Kernel description

n

Wind Kernel at the heart of VxWorks

Scalable Micro-Kernel Multi-tasking Pre-emptive priority based scheduling Optional round-robin (fair) scheduling inter task communication (messages and semaphores) and synchronization Fast context switching Low interrupt latency Fast interrupt response time Nested interrupts support 256 priorities

7/23/01

2001 Wind River Systems, Inc.

21

VxWorks Certification Strategy

n

Plan: Reverse Engineer VxWorks version 5.4 to meet the objectives of RTCA/DO-178B, Level A
VxWorks subset API rationale guidelines
FAA guidelines to Level A objectives as defined by RTCA/DO-178B Requirements from RTCA/DO-255 and ARINC 653 taken into consideration API of the subset remains consistent with VxWorks Functions compromising predictability and leading to memory fragmentation are eliminated

7/23/01

2001 Wind River Systems, Inc.

22

VxWorks Certification Strategy

n

Start with examination of the source code and architecture

determine functions which are predictable and certifiable eliminate unnecessary functionality and any features that may compromise a safety-critical application

Define a true subset of VxWorks that may be certified

removed:
network protocol support and file systems shared memory for multiple processors Object-oriented features: Dynamic links, other C++ features Debug facilities, BSPs, and various tools Dynamic allocation and de-allocation of memory

7/23/01

2001 Wind River Systems, Inc.

23

VxWorks Certification Strategy

n

Create a subset definition and rationale

results in a scaled-down version of VxWorks
15K SLOC

Create Software Hazard Analysis

Identifies potential failure conditions in the software, their potential impact, and proposed mitigation updated at each phase of the software lifecycle

Create a Plan for Software Aspects of Certification (PSAC) that describes the reverse engineering strategy
Provides the Certification Authorities an overview of the means of compliance and insight into the planning aspects for delivery of the product

7/23/01

2001 Wind River Systems, Inc.

24

VxWorks Certification Strategy

n n

Reverse Engineering Approach: Meet all 66 objectives of DO178B, Level A Reverse Engineer = Planning, Requirements, Design, Code, Tests, SCM and SQA
Existing Software Life Cycle Items:
Fully functional VxWorks software (source code and objects) Design documentation in the form of headers and comments Configuration management and corporate SQA policy

Configuration Items to be reverse engineered:

Software Quality Assurance Plan Defines the SQA process and activities Software Configuration Management Plan Defines the CM system and change control process.

7/23/01

2001 Wind River Systems, Inc.

25

Code Exists - Requirements re-engineered

Requirements

Design

2 1

Code

Test
7/23/01 2001 Wind River Systems, Inc. 26

VxWorks Certification Strategy

n

Configuration Items to be reverse engineered (cont):

Software Development Plan
Defines the processes used for requirements analysis, development, and test for the software product. Includes the standards for requirements, design, and code. Includes: Software Requirements Standards Software Design Standards Software Coding Standards

Software Verification Plan

Defines the test philosophy, test methods and approach to be used to verify the software product

Software Requirements Specification

Defines the high-level requirements applicable to the certifiable VxWorks subset

7/23/01

2001 Wind River Systems, Inc.

27

VxWorks Certification Strategy

n

Configuration Items to be reverse engineered (cont):

Tool Requirements Document
Defines the required functional behavior of a verification tool under normal operating conditions Verification Tool to support MCDC testing

7/23/01

2001 Wind River Systems, Inc.

28

VxWorks Certification Strategy

n

Configuration Items to be reverse engineered (cont):

Software Design Document
Describes the design of the certifiable VxWorks subset components Use off-the-shelf tools to examine the software design

Software Development Folder

Software Development Folder includes as a minimum: Reference to the applicable requirements Reference to the implementation (Design & Code) Evidence of reviews for the Requirements, Design, Code, and Test procedures Software Test Procedures Software Test Results Change History (CM System) Applicable Problem Reports

7/23/01

2001 Wind River Systems, Inc.

29

VxWorks Certification Strategy

n

Configuration Items to be reverse engineered (cont):

Version Description Document with Software Configuration Index and Software Life Cycle Environment Configuration Index
Identifies the components of the certifiable VxWorks subset with version information necessary to support regeneration of the product. Includes the tools used to build and test VxWorks image

Tool Qualification Document

Documents the qualification evidence for a verification tool against the requirements established in the PSAC and Tool Requirements Document

Traceability Matrix
Provides traceability from the requirements, to implementation, to test for the delivered software product

7/23/01

2001 Wind River Systems, Inc.

30

VxWorks Certification Strategy

n

Configuration Items to be reverse engineered (cont):

Software Accomplishment Summary
Documents the actual versus planned (per PSAC) activities and results for the project. Provides a summary of the means of compliance used for the software.

Sources
Provides the Source files for: Certifiable VxWorks subset Test Procedures Build and Test Scripts

Results
Documents the results of the functional and structural coverage testing. This includes the actual results and any applicable analyses performed including coverage analysis

7/23/01

2001 Wind River Systems, Inc.

31

VxWorks Certification Strategy

n

Approval Process
Wind River delivers all evidence of DO-178B compliance with its operating system and tools Relationship with Application Developer DER or regional FAA office Application Developer begins the DO-178B certification process for the application (PSAC) Customer builds application around the OS with the restriction that no sources are modified Attempts to build a modified image will result in compile or link errors Customer certifies its application under a TSO or STC
Wind River OS is certified along with the application

Wind River will defend its certification materials during any audits

7/23/01

2001 Wind River Systems, Inc.

32

VxWorks Certification Strategy

n

Current Status
FAA audits passed and complete as of July 17, 2001 Currently working on extensions to OS and BSP certification Work on VxWorks AE certification has begun

Sources: DO-178B, Wind River and Verocel, Inc

.
2001 Wind River Systems, Inc. 33

7/23/01