22-6-2014 The difference between Verification and Validation | Serendipity

http://www.easterbrook.ca/steve/2010/11/the-difference-between-verification-and-validation/ 1/5
Search:
Search
Serendipity
Applying systems thinking to computing, climate and sustainability
Menu
Search:
Search
About Steve
Moderation Policy
Popular Posts
Sitemap (blog contents)
The difference between Verification and Validation
29. November 2010 12 comments Categories: climate modeling
Sometime in the 1990s, I drafted a frequently asked question list for NASAs IV&V facility. Heres what I wrote on the meaning of the terms validation and verification:
The terms Verification and Validation are commonly used in software engineering to mean two different types of
analysis. The usual definitions are:
Validation: Are we building the right system?
Verification: Are we building the system right?
In other words, validation is concerned with checking that the system will meet the customers actual needs, while verification is concerned with whether the
system is well-engineered, error-free, and so on. Verification will help to determine whether the software is of high quality, but it will not ensure that the system is
useful.
The distinction between the two terms is largely to do with the role of specifications. Validation is the process of checking whether the specification captures the
customers needs, while verification is the process of checking that the software meets the specification.
Verification includes all the activities associated with the producing high quality software: testing, inspection, design analysis, specification analysis, and so on. It is
a relatively objective process, in that if the various products and documents are expressed precisely enough, no subjective judgements should be needed in order
to verify software.
In contrast, validation is an extremely subjective process. It involves making subjective assessments of how well the (proposed) system addresses a real-world
need. Validation includes activities such as requirements modelling, prototyping and user evaluation.
In a traditional phased software lifecycle, verification is often taken to mean checking that the products of each phase satisfy the requirements of the previous
phase. Validation is relegated to just the begining and ending of the project: requirements analysis and acceptance testing. This view is common in many software
engineering textbooks, and is misguided. It assumes that the customers requirements can be captured completely at the start of a project, and that those
requirements will not change while the software is being developed. In practice, the requirements change throughout a project, partly in reaction to the project
itself: the development of new software makes new things possible. Therefore both validation and verification are needed throughout the lifecycle.
Finally, V&V is now regarded as a coherent discipline: Software V&V is a systems engineering discipline which evaluates the software in a systems context,
relative to all system elements of hardware, users, and other software. (from Software Verification and Validation: Its Role in Computer Assurance and Its
Relationship with Software Project Management Standards, by Dolores R. Wallace and Roger U. Fujii, NIST Special Publication 500-165)
22-6-2014 The difference between Verification and Validation | Serendipity
http://www.easterbrook.ca/steve/2010/11/the-difference-between-verification-and-validation/ 2/5
Having thus carefully distinguished the two terms, my advice to V&V practitioners was then to forget about the distinction, and think instead about V&V as a toolbox, which
provides a wide range of tools for asking different kinds of questions about software. And to master the use of each tool and figure out when and how to use it. Heres one of
my attempts to visualize the space of tools in the toolbox:
A range of V&V techniques. Note that "modeling" and "model checking" refer to building and
analyzing abstracted models of software behaviour, a very different kind of beast from
scientific models used in the computational sciences
For climate models, the definitions that focus on specifications dont make much sense, because there are no detailed specifications of climate models (nor can there be
theyre built by iterative refinement like agile software development). But no matter the toolbox approach still works; it just means some of the tools are applied a little
differently. An appropriate toolbox for climate modeling looks a little different from my picture above, because some of these tools are more appropriate for real-time control
systems, applications software, etc, and there are some missing from the above picture that are particular for simulation software. Ill draw a better picture when Ive finished
analyzing the data from my field studies of practices used at climate labs.
Many different V&V tools are already in use at most climate modelling labs, but there is room for adding more tools to the toolbox, and for sharpening the existing tools
(what and how are the subjects of my current research). But the question of how best to do this must proceed from a detailed analysis of current practices and how effective
they are. There seem to be plenty of people wandering into this space, claiming that the models are insufficiently verified, validated, or both. And such people like to
pontificate about what climate modelers ought to do differently. But anyone who pontificates in this way, but is unable to give a detailed account of which V&V techniques
climate modellers currently use, is just blowing smoke. If you dont know whats in the toolbox already, then you cant really make constructive comments about whats
missing.
Share this:
Pinterest Facebook 56 Twitter 6 Google Reddit Digg StumbleUpon LinkedIn 22 Tumblr Print Email
12 Comments
1. Pingback: Tweets that mention The difference between Verification and Validation | Serendipity -- Topsy.com
2. Pingback: Validating Climate Models | Serendipity
3.
John Mueller
October 5, 2011 at 1:01 pm
Great article. Thank you!
4.
Molly Levy
August 8, 2012 at 8:28 am
This was a very useful description. The diagram was especially useful in analyzing verification vs validation testing.
5.
dimple kaushal
August 24, 2012 at 12:32 am
thnx for this article. . .this description with diagrams help me to cmplt my assignment. . .
1 A " T # P
22-6-2014 The difference between Verification and Validation | Serendipity
http://www.easterbrook.ca/steve/2010/11/the-difference-between-verification-and-validation/ 3/5
6.
Omama
September 21, 2012 at 2:27 pm
very nice and really helpful article it gives a clear idea of verification and validation.
Thanks..!!
7. Pingback: The difference between Verification and Validation | Nicole's world
8.
intansu
March 5, 2014 at 12:35 am
useful article on verification and validation. thanks!
9.
Andre Paulino de Lima
March 5, 2014 at 3:04 pm
tks for this very interesting and informative article
10.
Repson January
March 10, 2014 at 6:12 am
Thanks for this article. This will be helpful in my final year project.
11.
poonam
March 17, 2014 at 8:06 pm
very useful article thanks
12.
Nirvanie Prashad
April 2, 2014 at 12:00 pm
need it for the list of reference for my final year project. thank you.
13.
Samia Shah
May 25, 2014 at 1:53 am
very useful article . thank you
14.
Mohamed Bile Super
June 3, 2014 at 4:52 pm
Useful post. It helped alot to understand the differences between Validation and verification.
Thanks again for this awesome article.
15.
22-6-2014 The difference between Verification and Validation | Serendipity
http://www.easterbrook.ca/steve/2010/11/the-difference-between-verification-and-validation/ 4/5
sunil sahu
June 4, 2014 at 12:13 pm
This is the best different of validation and verification so read care fully
Thanks
Join the discussion:
Enter your comment here...
Fresh Serendipity:
From Computational Thinking to Systems Thinking June 6, 2014
TEDx talk: Should we trust climate models? May 23, 2014
Skills needed for a successful PhD April 7, 2014
Climate Model Bake-Off March 5, 2014
Weather balloons vs. climate balloons February 21, 2014
Climate Model vs. Satellite Data February 14, 2014
What Does the New IPCC Report Say About Climate Change? October 8, 2013
The Climate as a System, part 5: clouds September 3, 2013
The Climate as a System, part 4: earth system feedbacks September 2, 2013
The Climate as a System, part 3: greenhouse gases August 27, 2013
The Climate as a System, part 2: energy consumption August 26, 2013
The Climate as a System, part 1: the central equilibrium loop August 22, 2013
Why Systems Thinking? August 20, 2013
The Need for T-shaped Students August 15, 2013
The Internet: Saving Civilization or Trashing the Planet? April 10, 2013
Recent Tweets:
"Five reasons scientists do not like the
consensus on climate change"
feedly.com/e/TvfZOIt3
Retweeted by Steve Easterbrook
O. Bothe
@geschichtenpost
Expand
Ready for a party!
pic.twitter.com/QEBLvrddmQ
Steve Easterbrook
@SMEasterbrook
Show Photo
First official #ClimateChange refugees
evacuate their island homes for good
earthfirstjournal.org/newswire/2014/ via
@efjournal & @tveitdal
Retweeted by Steve Easterbrook
Richard McLellan
@RichardMcLellan
Expand
7h
21 Jun
21 Jun
Archives:
Select Month
Topics
Select Category
Blogroll
A Few Things Ill-Considered
All Models Are Wrong
Alternative Energy Action Network
Azimuth
Capital Climate
Climate Charts and Graphs
22-6-2014 The difference between Verification and Validation | Serendipity
http://www.easterbrook.ca/steve/2010/11/the-difference-between-verification-and-validation/ 5/5
Climate Denial Crock of the Week
Climate Lab Book
ClimateProgress
Crock of the Week
Deltoid
DeSmogBlog
Global Warming Art
Greenfyre's
Gristmill
http://www.ecoequity.org/
James Empty Blog
Left as an Exercise
Mind of Dan
Monbiot.com
More Grumbine Science
Only in it for the Gold
People and Place
RealClimate
Significant Figures
The Cost of Energy
The Frog that Jumped Out
The Intersection
The Way Things Break
Yale Climate Forum
Meta
Register
Log in
Entries RSS
Comments RSS
WordPress.org
2014 Serendipity. All rights reserved.
Design by picomol. Powered by WordPress.