Vous êtes sur la page 1sur 24

Ren Witte

Chapter 11 Key Points

Requirements Quality Assurance


Requirements
Inspections and
Reviews
Requirements Inspection
Process
SOEN 6481, Winter 2015/16 Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading
Ren Witte
Department of Computer Science
and Software Engineering
Concordia University
11.1

Key Points Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Next step after requirements documentation Process
Inspection Guidelines

Check if documented requirements satisfy certain qualities Verification of Z


Specifications
Might reveal defects in the specification Model Checking and
Theorem Proving
Requirements QA consists of: Consistency checks
Validity Checks
detecting defects, Inference

reporting them, Challenges with Formal


Methods

analysing their cause, Tool Support


Example: L4.verified
and fixing them Decision Table Verification

Natural Language
Result of this phase is a consolidated requirements document. Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.2

Outline Ren Witte

1 Requirements Inspections and Reviews


Requirements Inspection Process Key Points

Inspection Guidelines Requirements


Inspections and
Reviews

2 Verification of Z Specifications Requirements Inspection


Process
Inspection Guidelines

3 Model Checking and Theorem Proving Verification of Z


Specifications
Consistency checks Model Checking and
Validity Checks Theorem Proving
Consistency checks

Inference Validity Checks


Inference
Challenges with Formal Methods Challenges with Formal
Methods
Tool Support Tool Support

Example: L4.verified
Example: L4.verified
Decision Table Verification

Decision Table Verification Natural Language


Verification
Inspection Checklists
4 Natural Language Verification Automatic Assessment

Inspection Checklists Notes and Further


Reading
Automatic Assessment

5 Notes and Further Reading

11.3
Requirements Quality Assurance Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Copyright 2009 by John Wiley & Sons Ltd, [vL09] Reading

11.4

Motivation Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

Copyright 2003 by Pearson Education, [LW03]

11.5

The Requirements Challenge Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

From Ferguson & Lami, Automated Natural Language Analysis of Requirements, http://www.incose.org/delvalley/data/INCOSE- preview- QuARS_21June05.ppt

11.6
Overview Ren Witte

Key Points

Requirements
Inspections and
Reviews
Techniques for Requirements Quality Assurance Requirements Inspection
Process
Inspection Guidelines

Inspections and Reviews: Auditing of the specification with a dedicated inspection Verification of Z

team (similar to code reviews) Specifications

Model Checking and


Queries on a Requirements Database: When using RE tools, queries for structural Theorem Proving
Consistency checks
consistency checks become possible Validity Checks
Inference

Animation-Based Validation: Visualize (semi-)formal requirements using a tool Challenges with Formal
Methods

(e.g., for state machines) Tool Support


Example: L4.verified

Formal Verification: Check consistency, completeness, invariant properties, etc. Decision Table Verification

Natural Language
using formal methods Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.7

The Requirements Inspection Process (I) Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Copyright 2009 by John Wiley & Sons Ltd, [vL09]
Inspection Guidelines

Verification of Z
Specifications

Steps Model Checking and


Theorem Proving
Consistency checks

Inspection Planning: Determine Validity Checks


Inference

size and members of inspection team Challenges with Formal


Methods

schedule and scope of review meetings Tool Support


Example: L4.verified
format of inspection reports Decision Table Verification

Individual Reviewing: Review by each inspector (alone): Natural Language


Verification

Free mode, scope up to the reviewers discretion


Inspection Checklists
Automatic Assessment

Checklist based, using given list of questions and issues Notes and Further
Reading
Process based, using roles such as domain expert or developer

11.8

The Requirements Inspection Process (II) Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Copyright 2009 by John Wiley & Sons Ltd, [vL09]


Verification of Z
Specifications

Model Checking and


Theorem Proving
Steps Consistency checks
Validity Checks

Defect Evaluation: Review meetings to


Inference
Challenges with Formal
Methods
agree on defects (discard false positives) Tool Support

analyse agreed defects (including causes in the RE process)


Example: L4.verified
Decision Table Verification

recommend actions, document conclusions in inspection report Natural Language


Verification
this is not a problem solving session Inspection Checklists
Automatic Assessment
RD Consolidation: Revision of RS to address all concerns in inspection report Notes and Further
Reading

11.9
Inspection Guidelines Ren Witte

Key Points

Requirements
Inspections and
Follow the WHATWHOWHENWHERE guidelines Reviews
Requirements Inspection
Process
WHAT? Report should contain only facts, not opinions. Constructive, not Inspection Guidelines

offensive to RS authors. Use common structure for all inspectors. Verification of Z


Specifications

WHO? Inspectors independent from authors. Include different stakeholders Model Checking and
Theorem Proving
(e.g., domain expert, developer, end user). May include QA specialist Consistency checks

and RS authors. Minimum size of three. Validity Checks


Inference

WHEN? Not too early (RS still under development); not too late (design & Challenges with Formal
Methods

implementation underway). Shorter, more frequent meetings rather Tool Support


Example: L4.verified

than few, long ones. Decision Table Verification

Natural Language
WHERE? Critical system aspects (e.g., safety/security), places with already Verification
Inspection Checklists
detected defects (and areas impacting/impacted by them). Automatic Assessment

Notes and Further


Reading

11.10

Outline Ren Witte

Key Points

Requirements
Inspections and
1 Requirements Inspections and Reviews Reviews
Requirements Inspection
Process
Inspection Guidelines

2 Verification of Z Specifications Verification of Z


Specifications

Model Checking and


3 Model Checking and Theorem Proving Theorem Proving
Consistency checks
Validity Checks
Inference

4 Natural Language Verification Challenges with Formal


Methods
Tool Support
Example: L4.verified

5 Notes and Further Reading Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.11

Verification of Z Specifications (I) Ren Witte

Key Points

Checklist for Z (examples) Requirements


Inspections and
From van Lamsweerde [vL09, Chapter 5]: Reviews
Requirements Inspection
Process
Is there an initialization schema for every data schema? Inspection Guidelines

If this variable is declared as a partial function in this data schema, is there an Verification of Z
Specifications
invariant in the schema to specify the input domain where this function is Model Checking and

everywhere defined? Theorem Proving


Consistency checks
Validity Checks
Is this pre-condition consistent with invariants stated in the imported data Inference

schemas? Challenges with Formal


Methods
Tool Support
Does every imported variable in this -operation schema have a post-condition Example: L4.verified

to define its final state after the operation is applied?


Decision Table Verification

Natural Language
Is there a corresponding exception schema for the case where this Verification
Inspection Checklists

pre-condition does not hold? Automatic Assessment

Notes and Further


Many of these checks can be automated. Reading

11.12
Verification of Z Specifications (II) Ren Witte

Key Points

Requirements
Inspections and
Reviews
Syntax Checking Requirements Inspection
Process
Inspection Guidelines

AddBirthday Verification of Z
Specifications
BirthdayBook Model Checking and

name? : NAME Theorem Proving


Consistency checks

date? : DATE Validity Checks


Inference
Challenges with Formal
name? : known Methods
Tool Support
birthday 0 = birthday {name? 7 date?} Example: L4.verified
Decision Table Verification

Natural Language
Verification
(declaration symbols cannot appear in unquantified predicates) Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.13

Verification of Z Specifications (III) Ren Witte

Key Points

Requirements
Inspections and
Type Checking Reviews
Requirements Inspection
Consider the post-conditions: Process
Inspection Guidelines

OnLoan0 = OnLoan bc? Verification of Z


Specifications
BorrowedBy 0 = BorrowedBy {bc? p?} Model Checking and
Theorem Proving
together with the declarations: Consistency checks
Validity Checks

bc? : BookCopy Inference


Challenges with Formal

OnLoan : PBookCopy Methods


Tool Support

BorrowedBy : BookCopy
7 Person Example: L4.verified
Decision Table Verification

Natural Language
Verification
Errors? Inspection Checklists

Can you identify the type errors? Automatic Assessment

Notes and Further


Reading

11.14

Verification of Z Specifications (IV) Ren Witte

Key Points

Requirements
Inspections and
Static Semantics Checking Reviews
Requirements Inspection
Process

each variable must be declared Inspection Guidelines

Verification of Z
each variable must have specified initial value Specifications

Model Checking and


variables must be used within their scope Theorem Proving
Consistency checks

can be automatically checked through Zs import and initialization mechanisms Validity Checks
Inference
Challenges with Formal

Circularity Checking
Methods
Tool Support

No variable can be defined in terms of itself Example: L4.verified


Decision Table Verification

Available = Available \ {bc?} Natural Language


Verification
Inspection Checklists
can be automatically detected by finding cycles in the dependency graph Automatic Assessment

Notes and Further


Reading

11.15
Tool Support Ren Witte

The fuzz type checker for Z


Collection of (open source) tools for Key Points

Requirements
formatting Z specifications (using LATEX) Inspections and
Reviews

compliance checking (scope and type rules) Requirements Inspection


Process
Inspection Guidelines

Verification of Z
Input Example Specifications

Model Checking and


Theorem Proving
\begin{schema}{Club} Consistency checks
members: \power PERSON \\ Validity Checks
Inference
president: PERSON
Challenges with Formal
\where Methods
Tool Support
president \subseteq members Example: L4.verified
\end{schema} Decision Table Verification

Natural Language
\begin{schema}{Enrol} Verification
Inspection Checklists
\Delta Club \\ Automatic Assessment
new?: PERSON
Notes and Further
\where Reading
members = members \cup new? \\
president = president
\end{schema}

11.16

Formatted Output Ren Witte

Let PERSON be the set of all people:

[PERSON]. Key Points

Requirements
Inspections and
A club has a set of members and a president, who is one of the members: Reviews
Requirements Inspection
Process

Club Inspection Guidelines

Verification of Z
members : P PERSON Specifications

president : PERSON Model Checking and


Theorem Proving
Consistency checks
president members Validity Checks
Inference
Challenges with Formal
Methods

To enroll somebody in the club, we just add them to the set of members: Tool Support
Example: L4.verified
Decision Table Verification

Enrol Natural Language


Verification
Club Inspection Checklists

new? : PERSON Automatic Assessment

Notes and Further


Reading
members0 = members new?
president 0 = president

Post-condition: The president doesnt change when a new member is enrolled.


11.17

Type Checking Ren Witte

Key Points
Errors in Example Schemas Requirements
Inspections and
The fuzz type checker will identify a number of issues: Reviews
Requirements Inspection
Process

$> fuzz example.tex Inspection Guidelines

"example.tex", line 46: Type mismatch in Verification of Z


Specifications
left argument of infix relation Model Checking and
Theorem Proving
> Predicate: president \subseteq members Consistency checks

> Arg type: PERSON Validity Checks


Inference
> Expected: P ? Challenges with Formal
Methods
Tool Support
Example: L4.verified
"example.tex", line 54: Right argument of Decision Table Verification

operator \cup has wrong type Natural Language


Verification
> Expression: members \cup new? Inspection Checklists
Automatic Assessment
> Arg type: PERSON
Notes and Further
> Expected: P PERSON Reading

11.18
Report of defined names and types Ren Witte

fuzz also computes all direct and inferred types in a specification:


Key Points
$> fuzz -t example.tex
Requirements
Inspections and
Given PERSON Reviews
Requirements Inspection
Process

Schema Club Inspection Guidelines

members: P PERSON Verification of Z


Specifications
president: PERSON
Model Checking and
End Theorem Proving
Consistency checks

Schema \Delta Club Validity Checks


Inference
members: P PERSON Challenges with Formal
president: PERSON Methods
Tool Support
members: P PERSON Example: L4.verified
president: PERSON Decision Table Verification

End Natural Language


Verification
Inspection Checklists
Schema Enroll Automatic Assessment

members: P PERSON Notes and Further


president: PERSON Reading

members: P PERSON
president: PERSON
new?: PERSON
End
11.19

Outline Ren Witte

1 Requirements Inspections and Reviews Key Points

Requirements
Inspections and

2 Verification of Z Specifications
Reviews
Requirements Inspection
Process
Inspection Guidelines

3 Model Checking and Theorem Proving Verification of Z


Specifications
Consistency checks Model Checking and
Validity Checks Theorem Proving
Consistency checks

Inference Validity Checks


Inference
Challenges with Formal Methods Challenges with Formal
Methods
Tool Support Tool Support

Example: L4.verified
Example: L4.verified
Decision Table Verification

Decision Table Verification Natural Language


Verification
Inspection Checklists
Automatic Assessment
4 Natural Language Verification Notes and Further
Reading

5 Notes and Further Reading

11.20

Formal Verification Ren Witte

Key Points

Using Logic for Requirements Specification Requirements


Inspections and
Reviews
Propositional Logic Requirements Inspection
Process
Inspection Guidelines
Predicate Logic
Verification of Z
Specifications

Model Checking and


Recall the definitions Theorem Proving
Consistency checks

Satisfiability: A (propositional/predicate) formula p is satisfiable if there is some Validity Checks


Inference

interpretation I such that valI (p) = T . The satisfying interpretation I is Challenges with Formal
Methods

called a model of p, written I |= p Tool Support


Example: L4.verified

Validity: A formula p is valid, written |= p, if it is true (T ) in all interpretations Decision Table Verification

Natural Language
(i.e., p is a tautology). Verification
Inspection Checklists
Unsatisfiability: A formula p is unsatisfiable (or contradictory) if it is false (F ) in all Automatic Assessment

interpretations. Notes and Further


Reading

11.21
Application to RE? Ren Witte

Key Points

Requirements
Inspections and
Consistency check Reviews
Requirements Inspection
Are the requirements consistent? Check set of all requirements Process
Inspection Guidelines
S = {r1 , r2 , . . . , rn } = r1 r2 . . . rn (often also called the knowledge base): Verification of Z
if we can find a model I such that I |= S, our requirements are consistent Specifications

Model Checking and


if S is unsatisfiable, our requirements contain a contradiction Theorem Proving
Consistency checks

Consistency checks are supported by automated theorem provers. Validity Checks


Inference
Challenges with Formal
Methods
How to restore consistency? Tool Support
Example: L4.verified

Practice: Use the methods discussed in the lecture on requirements evaluation Decision Table Verification

Natural Language
Theory: Apply belief revision to find minimal change needed to restore Verification
Inspection Checklists
consistency in the knowledge base Automatic Assessment

Notes and Further


Reading

11.22

Logical Consequence Ren Witte

Validity Key Points

Often, we need to check if a certain assertion logically follows from a set of Requirements
Inspections and
requirements. Reviews
Requirements Inspection

E.g., from the collected requirements S we want to show that whenever the
Process
Inspection Guidelines

doors are open, the train is not moving: Verification of Z


Specifications

Model Checking and


= doorsOpen trainMoving Theorem Proving
Consistency checks
Validity Checks
Solution: check if S is valid, that is Inference
Challenges with Formal
Methods

|= S Tool Support
Example: L4.verified
Decision Table Verification

is a tautology. Natural Language


Verification
Inspection Checklists

Method Automatic Assessment

Notes and Further


Direct proof or refutation proof (resolution) Reading

Supported by automated theorem provers

11.23

Deriving New Requirements Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Generating new requirements through inference Verification of Z


Specifications
Inference procedures (such as resolution) allow us to automatically derive new Model Checking and
requirements from an existing set of requirements Theorem Proving
Consistency checks

can give important insights into the modeled domain Validity Checks
Inference

check with stakeholders (domain experts) if this is what they expect (if Challenges with Formal
Methods

necessary, translate back to natural language) Tool Support


Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.24
Challenge: Lack of World Knowledge Ren Witte

Key Points

Requirements
Inspections and
Common Problems Reviews

With large knowledge bases, a huge amount of statements can be automatically


Requirements Inspection
Process

generated
Inspection Guidelines

Verification of Z
not all of them will be insightful Specifications

Model Checking and


often, additional world knowledge is needed, but not encoded in the KB Theorem Proving
Consistency checks
E.g., we all know that books can be stored in a library, but that a library can not be Validity Checks

stored in a book
Inference
Challenges with Formal

but the theorem prover does not have such common sense knowledge, which can Methods
Tool Support
lead to spurious inconsistencies and irrelevant conclusions Example: L4.verified
Decision Table Verification
Encoding all this world knowledge is usually not feasible (in AI/Expert Systems this
Natural Language
is known as the knowledge acquisition bottleneck) Verification
Inspection Checklists
Requires requirements engineer trained in formal methods Automatic Assessment

Notes and Further


Reading

11.25

Tool Support Ren Witte

For RE, need automated tool support


Automated theorem prover: Key Points

Based (partly) on resolution theorem proving and unification Requirements


Inspections and
Interactive proofs (not fully automated) Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.26

Isabelle Ren Witte

Interactive Theorem Proving Framework Isabelle

Small logic core, can be used for propositional, predicate, higher-order logics Key Points

(HOL) Requirements
Inspections and
Reviews
Free/open source software (BSD license) Requirements Inspection
Process

Includes (higher-order) resolution and unification Inspection Guidelines

Verification of Z
Developed at University of Cambridge, Technical University Munich, and Specifications

Universit Paris-Sud Model Checking and


Theorem Proving

Has been used for many software verification projects Consistency checks
Validity Checks

Integrated with Emacs-based editor ProofGeneral Inference


Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.27
Isabelle/jEdit (Modern Version) Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.28
Copyright 2013 Gottfried Barrow, Isabelle Mailing List

ProofGeneral in Eclipse (under development) Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.29

Example: Verification of an operating system kernel Ren Witte

The L4.verified project


Proof of functional correctness of the seL4 OS microkernel with respect to a formal Key Points

specification Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

Copyright 2009 NICTA, http://ertos.nicta.com.au/research/l4.verified/approach.pml

11.30
The L4.verified project Ren Witte

Key Points
Some statistics (2009) Requirements
Inspections and
From http://ertos.nicta.com.au/research/l4.verified/ Reviews
Requirements Inspection
Code Size: 8,700 lines of C code plus 600 lines of ARM assembly code. Process
Inspection Guidelines

Verified Code Base: 7,500 lines of C code (excluding assembly code and boot Verification of Z
Specifications
loader) Model Checking and

Proof Size: 200,000 lines of Isabelle proof script. Theorem Proving


Consistency checks

Cost: 25-30 person years (including research, tool development, etc.).


Validity Checks
Inference

Authors estimate doing it again now would cost about 10 person Challenges with Formal
Methods

years. Tool Support


Example: L4.verified

Cost Comparison: Industry rule of thumb: software certification in the Common Decision Table Verification

Natural Language
Criteria process at Evaluation Level 6 is $10,000 per line of code Verification

(would come to $87m for seL4)


Inspection Checklists
Automatic Assessment

Detected Bugs: 160 bugs in the C code in total, of those 144 found with formal Notes and Further
Reading
verification

11.31

L4.verified Call Graph: Microkernel Complexity Ren Witte

The L4.verified project


Each C function in the kernel is a dot. If function A calls function B, there is an arrow
Key Points
from A to B. This is typical for performance-optimised microkernels. A snapshot Requirements
from early 2009: green dots are the C functions that had been verified at that time. Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.32
Copyright 2009 NICTA, http://ertos.nicta.com.au/research/l4.verified/visual.pml

Decision Table Check Ren Witte

Key Points

Requirements
QA for RS using decision tables Inspections and
Reviews

Count number of columns/conditions: for N entry conditions, there must be 2N Requirements Inspection
Process

columns Inspection Guidelines

Verification of Z
Otherwise table is incomplete Specifications

Model Checking and


If there are impossible conditions, this must be documented Theorem Proving
Consistency checks
Validity Checks
Inference

Train receives outdated acceleration command T T T T F F F Challenges with Formal


Methods

Train enters station block at speed X m.p.h. T T F F T F F Tool Support


Example: L4.verified

Preceding train is closer than Y meters T F T F F T F


Decision Table Verification

Natural Language
Verification
Full braking activated X X Inspection Checklists
Automatic Assessment
Alarm generated to station computer X X X X Notes and Further
Reading

11.33
Decision Table Check (II) Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Inconsistencies Verification of Z
Specifications
Two decision table entries are inconsistent if they define different actions for the
Model Checking and
same input conditions. Theorem Proving
Consistency checks
Validity Checks
Redundancies Inference

Two decision table entries are redundant if two columns have the same input- and
Challenges with Formal
Methods
Tool Support
output conditions. Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.34

Outline Ren Witte

1 Requirements Inspections and Reviews


Requirements Inspection Process Key Points

Inspection Guidelines Requirements


Inspections and
Reviews

2 Verification of Z Specifications Requirements Inspection


Process
Inspection Guidelines

3 Model Checking and Theorem Proving Verification of Z


Specifications
Consistency checks Model Checking and
Validity Checks Theorem Proving
Consistency checks

Inference Validity Checks


Inference
Challenges with Formal Methods Challenges with Formal
Methods
Tool Support Tool Support

Example: L4.verified
Example: L4.verified
Decision Table Verification

Decision Table Verification Natural Language


Verification
Inspection Checklists
4 Natural Language Verification Automatic Assessment

Inspection Checklists Notes and Further


Reading
Automatic Assessment

5 Notes and Further Reading

11.35

Defect-Based Checklists Ren Witte

Key Points

Requirements
Check for Defects in RD Inspections and
Reviews

Omission Is this acronym defined?


Requirements Inspection
Process
Inspection Guidelines
Are these definitions in the glossary? Verification of Z
If requirement relates to another, is this specified somewhere? Specifications

Requirements sufficient to ensure objective? Model Checking and


Theorem Proving

Contradiction Statement consistent with the system objectives and Consistency checks
Validity Checks

constraints? Inference
Challenges with Formal
Statement consistent with the other related statements? Methods
Tool Support

Inadequacy Does this RD item formulate what stakeholders really expect? Example: L4.verified
Decision Table Verification

Ambiguity Can statement be interpreted differently (readers from different Natural Language
Verification
context/background)? Inspection Checklists
Automatic Assessment
Other statements using same terms with different meanings?
Notes and Further
Reading

11.36
Defect-Based Checklists (contd.) Ren Witte

Key Points

Requirements
Inspections and
Reviews

Check for Defects in RD (II)


Requirements Inspection
Process
Inspection Guidelines

Immeasurability Is there a fit criterion associated with this quality requirement? Verification of Z
Specifications
Is the fit criterion measurable (quantities, protocol)? Model Checking and
Can test data be derived from this statement? Theorem Proving
Consistency checks

Noise Is this statement relevant for system objectives? Validity Checks


Inference
Does the negation of this statement make any sense? Challenges with Formal
Methods
Has this already been said? Tool Support
Example: L4.verified

Overspecification Does this statement entail premature design choice? Decision Table Verification

Would there be alternative sensible choices? Natural Language


Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.37

Defect-Based Checklists (contd.) Ren Witte

Key Points

Requirements
Inspections and
Reviews

Check for Defects in RD (III) Requirements Inspection


Process
Inspection Guidelines

Unfeasibility Is this implementable in view of infrastructure, budget, or time Verification of Z


Specifications
constraints? Model Checking and

Unintelligibility Will this be comprehensible to different stakeholders? Theorem Proving


Consistency checks

If not, why? Validity Checks


Inference

Poor Structuring Are statement structured in an apparent way? Challenges with Formal
Methods

Does an RD item cover unrelated requirements? Tool Support


Example: L4.verified

Does an RD item mix requirements and domain properties? Decision Table Verification

Unnecessary mix of functional and NF requirements? Natural Language


Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.38

Defect-Based Checklists (contd.) Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Check for Defects in RD (IV) Inspection Guidelines

Verification of Z
Forward Reference Is this concept, so far undefined, defined somewhere later? Specifications

Model Checking and


Remorse Has this concept been used already before this definition? Theorem Proving
Consistency checks

Poor Modifiability Would any change to a single RD item require changes Validity Checks
Inference
throughout the RS? Challenges with Formal
Methods

Opacity Are there implicit dependencies between RD items? Tool Support


Example: L4.verified
Decision Table Verification
More in the [vL09] textbook. Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.39
Other Checklists Ren Witte

Key Points

Requirements
Inspections and
Reviews
Quality-specific checklists Requirements Inspection

Checklists for specific non-functional requirements


Process
Inspection Guidelines

safety, security, performance, usability, . . . Verification of Z


Specifications

Model Checking and


Theorem Proving
Domain-specific checklists Consistency checks
Validity Checks
Check domain-specific concepts and operations (refinement of domain-independent Inference

checklists). Challenges with Formal


Methods
Tool Support
Example: L4.verified
Language-based checklists Decision Table Verification

Based on the specification language used (UML, templates, etc.). See [LW03, Natural Language
Verification
Chapter 29] for a number of checklists for UP artifacts. Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.40

Use Case Checklist Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

Anda, Bente, and Dag IK Sjberg. Towards an inspection technique for use case models. In Proceedings of the 14th international conference on Software engineering and knowledge engineering, pp. 127-134. ACM, 2002.
11.41

Outline Ren Witte

Key Points

Requirements
1 Requirements Inspections and Reviews Inspections and
Reviews
Requirements Inspection
Process

2 Verification of Z Specifications Inspection Guidelines

Verification of Z
Specifications

3 Model Checking and Theorem Proving Model Checking and


Theorem Proving
Consistency checks
Validity Checks

4 Natural Language Verification Inference


Challenges with Formal
Inspection Checklists Methods
Tool Support

Automatic Assessment Example: L4.verified


Decision Table Verification

Natural Language
Verification
5 Notes and Further Reading Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.42
Requirements in Practice Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Formal Inspection Guidelines
Specifications 10% Verification of Z
Specifications
Issues Model Checking and
Theorem Proving
Natural Language SRS are Consistency checks
Validity Checks
fragile & difficult to maintain Inference
Challenges with Formal

Limited tool support for Methods


Tool Support

90% Natural Language


Specifications
automatic analysis Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Industry distribution of SRS [MFI04] Automatic Assessment

Notes and Further


Reading

11.43

Automatic Assessment of Natural Language Specifications Ren Witte

QA on NL Specifications Key Points

QA on natural language specs: Requirements


Inspections and
Reviews
effective Requirements Inspection
Process
time-consuming (since performed manually) Inspection Guidelines

Verification of Z
hence costly (not very efficient) Specifications

Solution: (partially) automate QA for NL specifications Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Automated QA on NL Specifications Inference

Requires automated processing of natural language text


Challenges with Formal
Methods
Tool Support
using techniques from Natural Language Processing (NLP), Text Mining, and Example: L4.verified
Decision Table Verification
Semantic Computing Natural Language
Verification
We look at a few examples Inspection Checklists
Automatic Assessment
NASAs ARM Tool
Notes and Further
Reading
Semantic Assistants (Concordia)
ReqSAC (Concordia)

11.44

Natural Language Processing (NLP) Ren Witte

Key Points

Requirements
Definition Inspections and
Reviews
Requirements Inspection

Branch of Artificial Intelligence Process


Inspection Guidelines

Analysis of unstructured written text Verification of Z


Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Techniques Challenges Inference
Challenges with Formal

Named Entity Recognition NLP capitalizes on domain Methods


Tool Support

knowledge, but RE spans multiple Example: L4.verified

Linguistic Analysis Decision Table Verification


domains Natural Language
Word Sense Disambiguation Verification
Inspection Checklists

Information Retrieval Automatic Assessment

Notes and Further


Machine Learning Reading

11.45
NASA ARM Tool Ren Witte

Automatic Analysis of SRS documents Key Points


Developed at NASAs Software Assurance Technology Center (formerly at Requirements

http://satc.gsfc.nasa.gov/tools/arm/, discontinued in 2011) Inspections and


Reviews
Requirements Inspection
Computation of metrics for SRS Process
Inspection Guidelines

Micro-level (words) Verification of Z


Specifications
Macro-level (documents) Model Checking and
Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.46

ARM: Metrics Ren Witte

Micro-Level Indicators Key Points

imperatives (shall, is required to, must, . . .) Requirements


Inspections and
Reviews
continuances (below, as follows, :, listed, . . .) Requirements Inspection
Process
directives (e.g., i.e., Figure, Table, Note, . . .) Inspection Guidelines

Verification of Z
options (can, may, optionally, . . .) Specifications

weak phrases (adequate, effective, easy to, . . .) Model Checking and


Theorem Proving
Consistency checks
incompletes (TBD, but not limited to, as a minimum, . . .) Validity Checks
Inference
Challenges with Formal
Methods

Macro-Level Indicators Tool Support


Example: L4.verified

size of requirements Decision Table Verification

Natural Language
subjects (words preceding imperatives) Verification
Inspection Checklists

specification depth (numbering, section depth) Automatic Assessment

Notes and Further


readability Reading

Flesch Reading Ease index


Flesch-Kincaid Grade Level index

11.47

Readability Metrics Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process

Fog Index Inspection Guidelines

Verification of Z
Designed to measure the comprehension and retention of text. Indicates the Specifications

number of years of formal education a reader would need to understand a block of Model Checking and
Theorem Proving
text. Consistency checks
Validity Checks
Inference

Flesch Reading Ease Challenges with Formal


Methods

Originally developed to improve the readability of U.S. Navy technical documents, Tool Support
Example: L4.verified

the Kincaid metric is designed to measure the persistence and efficiency of text. Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.48
Readability Formulas Ren Witte

Key Points

Fog Requirements
Inspections and
Reviews

Fog = 0.4 (ASL + HW)


Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Where: Specifications

ASL = Average sentence length using number of words. Model Checking and
Theorem Proving
HW = Number of words with more than two syllables Consistency checks
Validity Checks
Inference
Flesch-Kincaid Challenges with Formal
Methods
Tool Support

Flesch-Kincaid = (0.39 ASL) + (11.8 ASW) 15.19 Example: L4.verified


Decision Table Verification

Natural Language
Where: Verification
Inspection Checklists
ASL = Average sentence length using number of words. Automatic Assessment

ASW = Average number of syllables per word. Notes and Further


Reading

11.49

ARM Statistics: Example Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.50

ARM: Mapping to IEEE 830 Quality Attributes Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.51
[William M. Wilson, Linda H. Rosenberg, Lawrence E. Hyatt, Automated Analysis of Requirement Specifications,, 19th International Conference on Software Engineering (ICSE97), 1997]
Semantic Assistants Ren Witte

Service-Oriented Architecture Key Points

Requirements
Services such as readability analysis are executed through a Web service Inspections and
Reviews
Results are displayed through client plug-ins (e.g., in OpenOffice, Eclipse, Requirements Inspection
Process

ReqWiki, . . .) Inspection Guidelines

Verification of Z
New clients and services can be added and improved independently Specifications

Model Checking and


Theorem Proving

Web Services for NLP


Consistency checks
Validity Checks
Focused Inference
Summarization Challenges with Formal
Methods
Calling an NLP Service Tool Support
Parameter Example: L4.verified
Decision Table Verification

NLP Service 1 Natural Language


Verification
NLP Service 2 Inspection Checklists
Client Server
Automatic Assessment

...
NLP Service n
Notes and Further
Reading
Word Processor
NLP Service
Result

11.52

Semantic Assistants Example: OpenOffice/LibreOffice Ren Witte

Readability Analysis Service


Check text for
Key Points
Readability Metrics (Flesch, Flesch-Kincaid, Fog) Requirements
Inspections and
Passive Voice sentences Reviews
Requirements Inspection
Process
... Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.53

Semantic Assistants Example: ReqWiki Ren Witte

Service results annotate Wiki pages


Check text for Key Points

Readability Metrics Requirements


Inspections and
Writing Quality Reviews
Requirements Inspection
Process
... Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.54
User Interface: Wiki-NLP Plug-in Ren Witte

Key Points

Requirements
Inspections and
Reviews
Input Text
Requirements Inspection
Process
Inspection Guidelines
Annotated
NLP results Verification of Z
Specifications

Model Checking and


Theorem Proving
NLP
Support Consistency checks
Validity Checks
Inference
Plugin
Challenges with Formal
Dialog
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.55

Semantic Assistants: Text Analysis Ren Witte

NLP Pipelines
Each service corresponds to an NLP Pipeline Key Points

Sequence of analysis components running on a text Requirements


Inspections and
Executed within GATE (General Architecture for Text Engineering), Reviews
Requirements Inspection

http://gate.ac.uk Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.56

Semantic Assistants: Passive Voice Example Ren Witte

Part-of-Speech Tagging
Add part-of-speech information (noun, verb, determiner, . . .)
Key Points

Verb Group Analysis Requirements


Inspections and
Analyse verb group for tense, voice, . . . Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.57
SA Service: Writing Quality Assessment Ren Witte

Key Points

Requirements
Features Inspections and
Reviews

Off-the-shelf After the Deadline tool Requirements Inspection


Process
Inspection Guidelines
Contextual spell, style & advance grammar checking Verification of Z
Specifications
Provides explanations & suggestions
Model Checking and
Theorem Proving
Detects unwanted passive voice Consistency checks
Validity Checks
Helps to improve text for other NLP services Inference
Challenges with Formal
Methods
Tool Support

Example Issues Example: L4.verified


Decision Table Verification

transactions must be valid before they are approved" Natural Language


Verification
Inspection Checklists
Rationale Automatic Assessment

Which parties (system, partner-application or stakeholders) are responsible for the Notes and Further
Reading
actions?

11.58

SA Service: Readability Assessment Ren Witte

Key Points

Requirements
Features Inspections and
Reviews
Requirements Inspection
Measures readability of text through Flesch & Kincaid metrics Process
Inspection Guidelines

SRS with poor scores are candidates for refactoring Verification of Z


Specifications

Model Checking and


Theorem Proving
Example Issues Consistency checks

The system shall configure the frame-rate as specified by the camera provided Validity Checks
Inference

during initialization & the frame-rate shall be capable of being automatically Challenges with Formal
Methods

modified based on bandwidth." [Flesh=0.8%, Kincaid=19.9pt] Tool Support


Example: L4.verified
Decision Table Verification

Refactoring Natural Language


Verification
The system shall set the frame-rate from the camera at startup." Inspection Checklists

The frame-rate shall be automatically modified based on bandwidth."


Automatic Assessment

Notes and Further


[Flesh=40.2%, Kincaid=50.1pt] Reading

11.59

SA Service: Requirement Quality Assurance Ren Witte

Key Points

Requirements
Features Inspections and
Reviews
Based on NASA ARM requirements metrics Requirements Inspection
Process

Uses linguistic patterns & keywords to detect:


Inspection Guidelines

Verification of Z
Option Defects give implementer latitude Specifications

Incompleteness missing concepts Model Checking and


Theorem Proving
Weak Phrases subject to multiple interpretations Consistency checks
Validity Checks
Inference
Challenges with Formal
Example Issues Methods
Tool Support

The system may approve customer & supplier requests in a timely manner" Example: L4.verified
Decision Table Verification

Natural Language
Rationale Verification
Inspection Checklists

may is ambiguous. Is request approval mandatory? For all stakeholders? Automatic Assessment

Notes and Further


timely manner is not quantified. Timely relative to who/what? Reading

11.60
ReqSAC (Requirements Specification Ambiguity Checker) Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Automatic Ambiguity Detection Process
Inspection Guidelines
MSc project at Concordia (I. Hussain [HOK07]): Verification of Z
Specifications
Find ambiguous requirements sentences using a classifier
Model Checking and
Based on machine learning: Theorem Proving
Consistency checks

examples for ambiguous sentences marked by human annotators Validity Checks


Inference
generation of a decision tree based on these examples Challenges with Formal
Methods
decision tree can then be used to find errors in new documents Tool Support
Example: L4.verified
Achieves about 87% accuracy Decision Table Verification

Natural Language
(Note: Not yet integrated into SA/ReqWiki.) Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.61

ReqSAC Decision Tree Example Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.62

ReqSAC Client Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Specifications

Model Checking and


Theorem Proving
Consistency checks
Validity Checks
Inference
Challenges with Formal
Methods
Tool Support
Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.63
Automatic Natural Language QA: Conclusions Ren Witte

Key Points

Use Natural Language Processing for better Quality&Efficiency Requirements


Inspections and
Reviews
Majority of specifications written today are using natural language Requirements Inspection
Process

Instead of performing only manual checks, add automated support: Inspection Guidelines

Verification of Z
needs technology from Natural Language Processing (NLP), Text Mining, and Specifications

Semantic Computing Model Checking and


Theorem Proving
Robust algorithms are already available Consistency checks

Need to be integrated with tools&processes deployed by software engineers Validity Checks


Inference

Will not replace inspection by a human requirements engineer (false positives, Challenges with Formal
Methods

missing semantic connections) Tool Support


Example: L4.verified

But can significantly reduce workload and allow humans to focus on the difficult
Decision Table Verification

Natural Language
issues Verification
Inspection Checklists

Prediction: will become standard tool in the next 35 years (comparable to spell Automatic Assessment

checkers etc. today) Notes and Further


Reading

11.64

Outline Ren Witte

1 Requirements Inspections and Reviews


Requirements Inspection Process Key Points

Inspection Guidelines Requirements


Inspections and
Reviews

2 Verification of Z Specifications Requirements Inspection


Process
Inspection Guidelines

3 Model Checking and Theorem Proving Verification of Z


Specifications
Consistency checks Model Checking and
Validity Checks Theorem Proving
Consistency checks

Inference Validity Checks


Inference
Challenges with Formal Methods Challenges with Formal
Methods
Tool Support Tool Support

Example: L4.verified
Example: L4.verified
Decision Table Verification

Decision Table Verification Natural Language


Verification
Inspection Checklists
4 Natural Language Verification Automatic Assessment

Inspection Checklists Notes and Further


Reading
Automatic Assessment

5 Notes and Further Reading

11.65

Reading Material Ren Witte

Required Key Points

[vL09, Chapter 5] (Requirements Quality Assurance)) Requirements


Inspections and
Reviews
[LW03, Chapter 29] (Assessing Requirements Quality, Checklists) Requirements Inspection
Process
Inspection Guidelines

Verification of Z
Supplemental Specifications

Model Checking and


[WRH97], [Lap09, pp.101109] (NASAs ARM tool) Theorem Proving
Consistency checks

[CL13] (reconstruction of NASAs ARM tool) Validity Checks


Inference

The L4.verified project, http://ertos.nicta.com.au/research/l4.verified/ Challenges with Formal


Methods
Tool Support
[HOK07] (ReqSAC) Example: L4.verified
Decision Table Verification
[SARW12] (Impact of NLP on RE Quality) Natural Language
Verification
Inspection Checklists
Automatic Assessment
Further Reading Notes and Further
Reading
[MFI04] (Market research on NLP tools for RE)
[Poh10, Part V, Chapters 2729] (Requirements Validation)

11.66
Tools Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
Process
Inspection Guidelines
Tool References
Verification of Z
Specifications
The fuzz type checker for Z, http://spivey.oriel.ox.ac.uk/mike/fuzz/
Model Checking and
The Isabelle Interactive Theorem Prover, Theorem Proving
Consistency checks

http://www.cl.cam.ac.uk/research/hvg/Isabelle/ Validity Checks


Inference

Semantic Assistants, http://www.semanticsoftware.info/semantic-assistants Challenges with Formal


Methods
Tool Support
ReqWiki, http://www.semanticsoftware.info/reqwiki Example: L4.verified
Decision Table Verification

Natural Language
Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.67

References I Ren Witte

Key Points
[CL13] Nathan Carlson and Phil Laplante. Requirements
The NASA automated requirements measurement tool: a Inspections and
Reviews
reconstruction. Requirements Inspection
Process

Innovations in Systems and Software Engineering, pages 115, 2013. Inspection Guidelines

http://dx.doi.org/10.1007/s11334-013-0225-8. Verification of Z
Specifications

[HOK07] Ishrar Hussain, Olga Ormandjieva, and Leila Kosseim. Model Checking and
Theorem Proving

Automatic Quality Assessment of SRS Text by Means of a Consistency checks


Validity Checks

Decision-Tree-Based Text Classifier. Inference


Challenges with Formal
In QSIC 07: Proceedings of the Seventh International Conference on Methods
Tool Support
Quality Software, pages 209218, Washington, DC, USA, 2007. IEEE Example: L4.verified

Computer Society.
Decision Table Verification

Natural Language
http://users.encs.concordia.ca/~kosseim/Publications/2007QSIC.pdf. Verification
Inspection Checklists

[Lap09] Phillip A. Laplante. Automatic Assessment

Notes and Further


Requirements Engineering for Software and Systems. Reading

Auerbach Publications, 1st edition, 2009.

11.68

References II Ren Witte

[LW03] Dean Leffingwell and Don Widrig.


Managing Software Requirements: A Use Case Approach. Key Points

Addison-Wesley, 2003. Requirements


Inspections and
Available online at http://clues.concordia.ca/record=b2529323. Reviews
Requirements Inspection
Process

[MFI04] Luisa Mich, Mariangela Franch, and Pierluigi Inverardi. Inspection Guidelines

Market research for requirements analysis using linguistic tools. Verification of Z


Specifications
Requirements Engineering, 9:4056, 2004. Model Checking and

http://dx.doi.org/10.1007/s00766-003-0179-8. Theorem Proving


Consistency checks
Validity Checks

[Poh10] Klaus Pohl. Inference


Challenges with Formal
Requirements Engineering: Fundamentals, Principles, and Techniques. Methods
Tool Support
Springer-Verlag Berlin Heidelberg, 2010. Example: L4.verified
Decision Table Verification

[SARW12] Bahar Sateli, Elian Angius, Srinivasan Sembakkam Rajivelu, and Ren Natural Language
Verification
Witte. Inspection Checklists

Can Text Mining Assistants Help to Improve Requirements


Automatic Assessment

Notes and Further


Specifications? Reading

In Mining Unstructured Data (MUD 2012), Kingston, Ontario, Canada,


October 17 2012.
http://www.semanticsoftware.info/system/files/mud2012-ReqWiki.pdf.
11.69
References III Ren Witte

Key Points

Requirements
Inspections and
Reviews
Requirements Inspection
[vL09] Axel van Lamsweerde. Process
Inspection Guidelines
Requirements Engineering: From System Goals to UML Models to Verification of Z
Software Specifications. Specifications

John Wiley & Sons, 2009. Model Checking and


Theorem Proving
Consistency checks

[WRH97] William M. Wilson, Linda H. Rosenberg, and Lawrence E. Hyatt. Validity Checks
Inference
Automated analysis of requirement specifications. Challenges with Formal
Methods

In Proceedings of the 19th International Conference on Software Tool Support


Example: L4.verified
engineering, pages 161171. ACM, 1997. Decision Table Verification

http://dx.doi.org/10.1145/253228.253258. Natural Language


Verification
Inspection Checklists
Automatic Assessment

Notes and Further


Reading

11.70

Vous aimerez peut-être aussi