Enterprise Risk Management and Ethics Readings PDF

2014
Energy Risk Professional

ERP Exam Course Pack
READINGS THAT ARE FREELY AVAILABLE ON THE GARP WEBSITE

The Enterprise Risk Management (ERM) and Business Ethics section of the 2014 ERP Study Guide
includes several additional readings from online sources that are freely available on the GARP
website (link to 2014 Online Readings). These readings include learning objectives that cover
specialized topics or current trends in Enterprise Risk Management that are unavailable in traditional
text books.
The 2014 ERP Examination will include questions drawn from the following AIMs for each reading:
Readings for Enterprise Risk Management and Business Ethics

Enterprise Risk Management (ERM) Framework
1.
COSO, Understanding and Communicating Risk Appetite. (January 2012).
Define risk appetite and explain the role of risk appetite in corporate governance.
Describe considerations a firm must make in determining its risk appetite, and explain
how an organizations risk appetite can differ for various risk factors.
Describe the objective and characteristics of an effective risk appetite statement.
Differentiate between risk appetite and risk tolerance, and explain how an organization can align its risk tolerance to its risk appetite.
Explain how an organization can develop, communicate, monitor and update its
risk appetite.
2.
COSO, Risk Assessment in Practice. (October 2012).
Describe the steps in the risk assessment process.
Compare impact, likelihood, vulnerability, and speed of onset of potential risk events
and explain how a scale can be created to assess these four factors with respect to
specific potential risk events.
Identify examples of actions a firm can take to reduce its vulnerability to specific
risk events.
Compare and contrast qualitative and quantitative measurement techniques in

assessing risks, and describe examples of each.
Describe methods to capture interactions between risk factors, including risk interaction maps and the bow-tie diagram.
Explain how a risk hierarchy can be created to rank and prioritize risks.
Explain the use of heat maps and MARCI (Mitigate, Assure, Redeploy, and Cumulative
Impact) charts in aggregating, comparing, and prioritizing risks faced by a firm.
2014 Global Association of Risk Professionals. All rights reserved.
2014
Energy Risk Professional

ERP Exam Course Pack
3.
COSO, Developing Key Risk Indicators to Strengthen Enterprise Risk Management.

(December 2010).
4.
Differentiate between key performance indicators (KPIs) and key risk indicators (KRIs).
Describe some qualities of effective KRIs.
Explain considerations and challenges faced by a firm in the process of developing KRIs.
Explain how a firm can gather information used in developing KRIs.
Describe how KRIs can be reported, monitored, updated and communicated.
Explain ways that KRIs can provide value to an organization.
Robert Bea, Ian Mitroff, Daniel Farber, Howard Foster and Karlene H. Roberts. A New
Approach to Risk: The Implications of E3. (Palgrave Macmillan 2009).
Describe the factors used to assess risk in a complex organization or system.
Understand the interdisciplinary aspect of modeling risk associated with a complex

system.
Identify and understand the implications of a Type Three Error (E3).
Summarize the elements used in a Complex Infrastructure System (CIS) risk

assessment.
Explain why human error may be overlooked in risk assessment and why engineering
analyses often underestimate the probability of a system failure.
GARP Code of Conduct

5.
Global Association of Risk Professionals (GARP). Code of Conduct.
Describe the responsibility of each GARP member with respect to professional

integrity, ethical conduct, conflicts of interest, confidentiality of information and
adherence to generally accepted practices in risk management.
Describe the potential consequences of violating the GARP Code of Conduct.
2014 Global Association of Risk Professionals. All rights reserved.
Thought Leadership in ERM | Enterprise Risk Management Understanding and Communicating Risk Appetite |
C o m m i t t e e
o f
S p o n s o r i n g
O r g a n i z a t i o n s
o f
t h e
T r e a d w a y
C o m m i s s i o n
Thought Leadership in ERM
E n ter p r i se
R i s k
M a n a g eme n t
Understanding and
Communicating Risk Appetite
By
Dr. Larry Rittenberg and Frank Martens
www.co s o.o rg
| Enterprise Risk Management Understanding and Communicating Risk Appetite | Thought Leadership in ERM
Authors
Dr. Larry Rittenberg
Ernst & Young Professor of Accounting
University of Wisconsin-Madison School of Business

Frank Martens
Director, PricewaterhouseCoopers (PwC)

COSO Board Members

David L. Landsittel
COSO Chair
Larry E. Rittenberg
COSO Chair - Emeritus
Mark S. Beasley/Douglas F. Prawitt

American Accounting Association
Chuck E. Landes
American Institute of CPAs (AICPA)
Richard F. Chambers
The Institute of Internal Auditors
Jeff C. Thomson
Institute of Management Accountants
Marie N. Hollein
Financial Executives International
Preface
This project was commissioned by COSO, which is dedicated to providing thought leadership
through the development of comprehensive frameworks and guidance on enterprise risk
management, internal control, and fraud deterrence designed to improve organizational
performance and governance and to reduce the extent of fraud in organizations. COSO is a
private sector initiative, jointly sponsored and funded by the following organizations:

American Accounting Association (AAA)
Financial Executives International (FEI)
The Institute of Management Accountants (IMA)
The Institute of Internal Auditors (IIA)
www.co s o.o rg
Committee of Sponsoring Organizations

of the Treadway Commission
www.co s o.o rg
E n ter p r i se
R i s k
M a n a g eme n t
Understanding and
Research Commissioned by
Co m m i t te e o f S p o n s o r i n g Organizations of the Treadway Commission
January 2012
Copyright 2012, The Committee of Sponsoring Organizations of the Treadway Commission (COSO).
1 2 3 4 5 6 7 8 9 0 PIP 198765432
All Rights Reserved. No part of this publication may be reproduced, redistributed, transmitted or displayed in any form or
by any means without written permission. For information regarding licensing and reprint permissions please contact the
American Institute of Certified Public Accountants, licensing and permissions agent for COSO copyrighted materials.
Direct all inquiries to copyright@aicpa.org or to AICPA, Attn: Manager, Rights and Permissions, 220 Leigh Farm Rd.,
Durham, NC 27707. Telephone inquiries may be directed to 888-777-7707.
www.co s o.o rg
Thought Leadership in ERM | Enterprise Risk Management Understanding and Communicating Risk Appetite | iii
Content Outline
Page
Executive Summary
Overview
Risk Appetite Statements
Risk Appetite and Risk Tolerance

Developing Risk Appetite
11
15
18
Monitoring and Updating Risk Appetite
20
Roles
21
Summary of Considerations
23
About COSO
24
About the Authors
24
www.co s o.o rg
Executive Summary
Organizations encounter risk every day as they pursue their
objectives. In conducting appropriate oversight, management
and the board must deal with a fundamental question: How
much risk is acceptable in pursuing these objectives? Added
to this, regulators and other oversight bodies are calling
for better descriptions of organizations risk management
processes, including oversight by the board.
This thought leadership document is one of a series
of papers, sponsored by the Committee of Sponsoring
Organizations of the Treadway Commission (COSO), to
help organizations implement enterprise risk management
(ERM). The COSO document Enterprise Risk Management
Integrated Framework explicitly states that organizations
must embrace risk in pursuing their goals. The key is to
understand how much risk they are willing to accept.
Further, how should an organization decide how much
risk it is willing to accept? To what extent should the risks
accepted mirror stakeholders objectives and attitudes
towards risk? How does an organization ensure that
its units are operating within bounds that represent the
organizations appetite for specific kinds of risk?
Risk appetite is the amount of risk, on a broad level,
an organization is willing to accept in pursuit of value.
Each organization pursues various objectives to add
value and should broadly understand the risk it is
willing to undertake in doing so.
These questions are embodied in the notion of an entitys

risk appetite. The objective of this paper is to help an
organization its senior management, board, and key
operating personnel to develop and communicate a clear
understanding of its risk appetite, both to determine which
objectives to pursue and to manage those objectives within the
organizations appetite for risk.
Many organizations view risk appetite as the subject of
interesting theoretical discussions about risk and risk
management, but do not effectively integrate the concept
into their strategic planning or day-to-day decision making.
We believe that discussions about applying risk appetite go
well beyond theory, and that when properly communicated,
risk appetite provides a boundary around the amount of
risk an organization might pursue. An organization with an
aggressive appetite for risk might set aggressive goals,
while an organization that is risk-averse, with a low appetite

for risk, might set conservative goals.
Similarly, when a board considers a strategy, it should
determine whether that strategy aligns with the
organizations risk appetite. When properly communicated,
risk appetite guides management in setting goals and
making decisions so that the organization is more likely to
achieve its goals and sustain its operations.
Enterprise Risk Management and Decision Making
ERM is not isolated from strategy, planning, or day-to-day

decision making. Nor is it about compliance. ERM is part of
an organizations culture, just as making decisions to attain
objectives is part of an organizations culture.
To fully embed ERM in an organization, decision makers

must know how much risk is acceptable as they consider
ways of accomplishing objectives, both for their organization
and for their individual operations (division, department,
etc.). For example, one CEO recently reported that his
organization needed to increase its risk appetite amid
expectations that key measures of its profitability would
fall or stagnate. A financial organization with a lower risk
appetite might choose to avoid opportunities that are more
risky, but offer greater returns. Finally, another organization
with a high risk appetite might decide to procure natural
resources from a volatile country where the total investment
could be wiped out at the whim of the political leader. The
rewards may be high, but so too may the risks. Organizations
make decisions like these all the time. Only if they clearly
think about their risk appetite can they balance risks and
opportunities.
An organization must consider its risk appetite at the same
time it decides which goals or operational tactics to pursue.
To determine risk appetite, management, with board review
and concurrence, should take three steps:
1. Develop risk appetite
2. Communicate risk appetite
3. Monitor and update risk appetite
These three steps are discussed briefly below, and in detail
in the body of this paper.
www.co s o.o rg
Develop Risk Appetite
Developing risk appetite does not mean the organization

shuns risk as part of its strategic initiatives. Quite the
opposite. Just as organizations set different objectives, they
will develop different risk appetites. There is no standard
or universal risk appetite statement that applies to all
organizations, nor is there a right risk appetite. Rather,
management and the board must make choices in setting
risk appetite, understanding the trade-offs involved in having
higher or lower risk appetites.
Communicate Risk Appetite
Several common approaches are used to communicate

risk appetite. The first is to create an overall risk appetite
statement that is broad enough yet descriptive enough
for organizational units to manage their risks consistently
within it. The second is to communicate risk appetite for
each major class of organizational objectives. The third is to
communicate risk appetite for different categories of risk.
Monitor and Update Risk Appetite
Once risk appetite is communicated, management, with

board support, needs to revisit and reinforce it. Risk
appetite cannot be set once and then left alone. Rather,
it should be reviewed in relation to how the organization
operates, especially if the entitys business model changes.
Management should monitor activities for consistency with
risk appetite through a combination of ongoing monitoring
and separate evaluations. Internal auditing can support
management in this monitoring. In addition, organizations,
when monitoring risk appetite, should focus on creating a
culture that is risk-aware and that has organizational goals
consistent with the boards.
Can It Be Done?
This is a common question. Its tone implies two things:

(1) articulating risk appetite is too difficult, and (2) risk is
considered when management sets strategies, and to further
communicate risk appetite is an exercise that simply adds
overhead and does not contribute to organizational growth.
Recent world events involving governments, businesses,
not-for-profit organizations, and the recent financial crisis
clearly show that having a communicated risk appetite
built into organizational activities could have preserved
a considerable amount of capital. We all know the costs
of failing to manage risk. Examples include the cost to
companies and travellers when air travel closed down
after a volcanic eruption in 2010 in Iceland; the cost of
the financial crisis to U.S. taxpayers, stockholders, and
debtholders; and the social cost of government budgets in
Greece, Spain, Ireland, and Portugal.
Perhaps organizations are still tied to the old-school thinking
that it will not happen here. The easy rebuttal is that it
has happened somewhere, so all organizations should
work to manage their risks within their risk appetite. Rather
than asking Can it be done? lets say Lets get it done.
Determining risk appetite is an element of good governance
that managements and boards owe to stakeholders.
Develop/
Revise
Risk
Appetite
Monitor
www.co s o.o rg
Communicate
Overview
Risk Appetite Is an Integral
Part of Enterprise Risk Management
COSOs Enterprise Risk Management Integrated
Framework defines risk appetite as follows:
The amount of risk, on a broad level, an entity is willing
to accept in pursuit of value. It reflects the entitys risk
management philosophy, and in turn influences the
entitys culture and operating style. Risk appetite
guides resource allocation. Risk appetite [assists the
organization] in aligning the organization, people, and
processes in [designing the] infrastructure necessary to
effectively respond to and monitor risks.1
This definition raises some important points. Risk appetite
is strategic and is related to the pursuit of
organizational objectives;
forms an integral part of corporate governance;
guides the allocation of resources;

guides an organizations infrastructure, supporting

its activities related to recognizing, assessing,
responding to, and monitoring risks in pursuit of
organizational objectives;
As an organization decides on its objectives and its

approach to achieving strategic goals, it should consider
the risks involved, and its appetite for such risks, as a basis
for making those important decisions. Those in governance
roles should explicitly understand risk appetite when
defining and pursuing objectives, formulating strategy, and
allocating resources. The board should also consider risk
appetite when it approves management actions, especially
budgets, strategic plans, and new products, services, or
markets (in other words, a business case).
In working towards their objectives, organizations choose
strategies and develop metrics to show them how close they
are to meeting those objectives. Managers are motivated to
achieve the objectives through reward and compensation
programs. The strategy is then operationalized by decisions
made throughout the organization. Decisions are made to
achieve the objectives (increase market share, profitability,
etc.). But achieving objectives also depends on identifying
risk and determining whether the risks are within the
organizations risk appetite.
influences the organizations attitudes towards risk;

is multi-dimensional, including when applied to the
pursuit of value in the short term and the longer term of
the strategic planning cycle; and
requires effective monitoring of the risk itself and of the
organizations continuing risk appetite.
COSO, Enterprise Risk Management Integrated Framework, p. 19.
www.co s o.o rg
Considerations Affecting Risk Appetite
Risk appetite is not developed in isolation from other

factors. An organization should consider its capacity to
take on extra risk in seeking its objectives. It should also
consider its existing risk profile, not as a determinant of

risk appetite but as an indication of the risks it currently
addresses. An overview of the considerations affecting risk
appetite is shown in Exhibit 1.
Exhibit 1
Overview of Considerations Affecting Risk Appetite
Existing
Risk Profile
The current level and distribution of risks across

the entity and across various risk categories
Risk
Capacity
The amount of risk that the entity is able to

support in pursuit of its objectives
Risk
Tolerance
Acceptable level of variation an entity is willing

to accept regarding the pursuit of its objectives
Attitudes
Towards Risk
The attitudes towards growth, risk, and return
There may be other factors to consider as well. Some

organizations may gauge how quickly their competitive
environment is changing. A telecommunications company,
for example, must anticipate how technology and user
preferences will affect product development, making a
relevant time frame important.
As an example of high risk appetite, a defense contractor
dealing in trucks decided that the risk of being behind
in technology was so large that it essentially bet the
company on developing a vehicle appropriate for the types
of wars occurring around the world. If the contractor had
been unsuccessful in procuring a new government order, it
would have been out of business. The risk appetite was high,
but it was understood by all involved in the process.
www.co s o.o rg
Determination
of
Risk
Appetite
However, the board was well aware of the risks, having

debated the issue extensively in board meetings, and it
concurred with managements decision (an acknowledgement
of risk appetite and the linkage of risk appetite and strategy).
The investing public was also aware because the nature of
the risks had been communicated (and the stock dropped to
historic lows). What is notable is that the risk was carefully
debated and the company was going to succeed or die
as opposed to almost certainly dying (slowly) if it did not take
on risk through an aggressive strategy.
The point is that risk and strategy are intertwined. One does
not exist without the other, and they must be considered
together. That consideration takes place throughout the
execution of the strategy, and it is most important when
strategy is being formulated with due regard for risk appetite.
An organization has a number of goals and objectives it

can pursue. Ultimately, it will decide on those that best
meet stakeholder preferences for growth, return, safety,
sustainability and its willingness to accept risk. The
objectives, in turn, may be pursued using a number of
alternative strategies. As shown in Exhibit 2, the articulation
of a risk appetite provides bounds on the choice of
strategies and the operational decisions that are made to
pursue those objectives.
One major problem that led to the current financial crisis was
that although objectives had been created, there was no
articulation of risk appetite or identification of those
responsible when risks were incurred.
Exhibit 2
Interrelationship of Strategy, Management Decisions, and Risk Appetite
Sets strategic
goal and
objectives
Formulates
strategies
Strategy 1
Strategy 2
Strategy 3
...
Establishes
operations,
compliance,
and reporting
objectives
Makes decisions
on how to manage
risks relating to
the achievement
of objectives
Considers risk appetite in setting of strategies, objectives, and how to manage risks
Steps in Adopting Risk Appetite
Each organization must determine its own risk appetite; there

is no single universal risk appetite. But how does an organization
get to the point of having a risk appetite statement that can be
communicated through the organization? And how does risk
appetite stay relevant over time?
To effectively adopt risk appetite, an organization must take
three key steps:
1. Management develops, with board review and

concurrence, a view of the organizations overall
risk appetite.
2. This view of risk appetite is translated into a written

or oral form that can be shared across the organization.
3. Management monitors the risk appetite over time,
adjusting how it is expressed as business and
operational conditions warrant.
These three steps will be discussed in detail in later sections
of this paper.
In a recent survey, less than half of the respondents said
they had a formal process for developing and
communicating risk appetite.2
Towers Watson, 2011 Risk and Finance Manager Survey
www.co s o.o rg

An organizations risk appetite should be articulated
and communicated so that personnel understand that
they need to pursue objectives within acceptable limits.
Without some articulation and communication, it is difficult
for management to introduce operational policies that
assure the board and themselves that they are pursuing
objectives within reasonable risk limits. A risk appetite
statement effectively sets the tone for risk management.
The organization is also more likely to meet its strategic
goals when its appetite for risk is linked to operational,
compliance, and reporting objectives.
The length of a risk appetite statement will vary by
organization. Some statements require several sentences
to express how much risk is acceptable, while others may

be more succinct and still clearly communicate
managements appetite for risk. The aim is to balance
brevity with the need for clarity.
Characteristics of Effective
A risk appetite statement is useful only if it is clear and

can be implemented across the organization. As we
noted earlier, risk appetite must relate to the pursuit of
organizational objectives and must start at the top. In
developing and evaluating a statement, the organization
should ensure that risk appetite (Exhibit 3)
Exhibit 3
Link to
Objectives
Facilitate
Monitoring of Risk
Facilitate
Alignment
Operations
Decisions
Time Frame,
Portfolio of Projects
Risk
Appetite
Determine
Acceptable Risk
Tolerances
People, Process,
Infrastructure
directly links to the organizations objectives;

is stated precisely enough that it can be communicated
throughout the organization, effectively monitored, and
adjusted over time;
helps with setting acceptable tolerances for risk,
thereby identifying the parameters of acceptable risks
(discussed in the next section);
facilitates alignment of people, processes, and
infrastructure in pursuing organizational objectives
within acceptable ranges of risk;
State With
Sufficient Precision
Communicate,
Monitor, Adjust
Specific
Objectives
facilitates monitoring of the competitive environment

and considers shareholders views in identifying
the need to reassess or more fully communicate the
risk appetite;
recognizes that risk is temporal and relates to the

time frame of the objectives being pursued; and

recognizes that the organization has a portfolio of

projects and objectives, as well as a portfolio of risks
to manage, implying that risk appetite has meaning at
the individual objective level and at the portfolio level.
Risk appetite should be descriptive enough to guide actions
across the organization. Management and the board should
determine whether compensation incentives are aligned with
risk appetite, not only for top management but throughout
the organization.
www.co s o.o rg
Thought Leadership in ERM | Enterprise Risk Management Understanding and Communicating Risk Appetite | 7
Reluctance to Embrace Risk Appetite
Some organizations are reluctant to develop and

communicate risk appetite. Others might argue that risk
management did not prevent the recent financial crisis and
thus question the usefulness of ERM in general. Others
believe that they have expressed their organizations risk
appetite in the normal course of business, and that
developing further risk appetite statements will not result
in any new approach to managing risk.
Such arguments can be misleading to management and
the board. To forgo discussion of an organizations risk
appetite is to assume that everyone will understand vague
comments. History shows that when risk appetite is not
considered (especially in compensation schemes),
the organization often suffers from greater risks than
anticipated. For example, had financial institutions clearly
communicated a risk appetite for unsecured mortgagebacked financial instruments, their management and
boards would have likely asked questions that would lead
to better risk identification, such as the following:
What if housing failures differ from the historical model?
What if mortgages fail systematically and are highly
correlated to an area we are investing in?
Could decisions made by some of our operational
personnel be creating risks that go beyond our
risk appetite?
Risk Appetites Are Not All the Same
Regulators and investors are calling for greater disclosure

of risk management processes so that shareholders can
better understand not only the risks an organization faces,
but the organizations appetite for risk and how it manages
(or accepts) that risk. For example, a mining company we
are aware of clearly identified its risk appetite and risk
mitigation procedures for operational risks. At the same
time, it decided it could not manage commodity price risk,
leaving stakeholders to decide how to consider that risk in
developing their portfolios.
To earn an adequate score for overall ERM from some rating
agencies, management must be able to articulate risk appetite
and assess and reconcile the appropriateness of individual risk
limits given to operational management.
Some companies embrace a high appetite for regulatory

risk believing that it will lead to greater profitability
because regulator fines were significantly lower than
the cost of mitigating the compliance risks. One company
ignored many health and safety regulations and fines when
incurred, but it did not fully understand the magnitude of
risks, such as the government shutting down its operations.
While the company had a high risk appetite for fines, its
lack of appreciation for the risk of shutdown led to a poorly
articulated and implemented risk appetite. Organizations
can choose to have high or low risk appetites, but those
appetites need to consider shareholder interests and the
type and magnitude of risks that the organization needs to
manage. We have no preference for a particular level of
appetite. Whatever the risk appetite is, it should be stated
clearly enough that it can be managed throughout
the organization, and reviewed by the board of directors.
www.co s o.o rg
8 | Enterprise Risk Management Understanding and Communicating Risk Appetite | Thought Leadership in ERM
Examples of Risk Appetite Statements
Risk appetite statements often start out broad and become

more precise as they cascade into departments and
operations across the organization. Some organizations
find that broad statements crafted around terms such
as low, medium, or high appetite meet the
characteristics of risk appetite statements listed above.
Others are more precise, making statements like We are
not comfortable accepting more than a 10% probability that
we will incur losses of more than a set dollar amount in
pursuit of a specific objective.
Which type of statement is best for a particular entity is a
management decision. Some organizations may find terms
like low appetite clear enough to be communicated
and monitored effectively within the organization.
However, such statements are vague and can be difficult
to communicate and implement. Often, as organizations
become more experienced in risk management, their risk
appetite statements will become more precise.
The following examples of risk appetite statements
illustrate the characteristics we identified above.
Health Care Organization: The following represents
one part of the health care organizations risk appetite

statement. The organization has specific objectives related
to (1) quality of customer care, (2) attracting and retaining
high-quality physicians and health researchers, and

(3) building sustainable levels of profit to provide access
to needed capital and to fund existing activities. The
statement starts as follows:
The Organization operates within a low overall risk range.
The Organizations lowest risk appetite relates to safety
and compliance objectives, including employee health
and safety, with a marginally higher risk appetite towards
its strategic, reporting, and operations objectives. This
means that reducing to reasonably practicable levels the
risks originating from various medical systems, products,
equipment, and our work environment, and meeting our legal
obligations will take priority over other business objectives.
In our view, this risk appetite statement does three
things effectively:
Communicates, with sufficient precision, that the
organization wants to sustain its business over a long
period of time
Expresses a low risk appetite in pursuing all the
organizations objectives
Expresses a very low appetite for risks associated
with employee safety and compliance
Business performance can be increased if capital and resources

are allocated more effectively, reflecting the balance of risks and
rewards in a more integrated and dynamic fashion. In that respect,
risk appetite can be considered the cornerstone of modern
approaches to bank management, such as value-based
management (VBM) and its various implementations. 3
IBM, Risk Appetite: A Multi-faceted Approach to Risk Management, April 2008.
www.co s o.o rg
University: The universitys main objective is to continue

as a preeminent teaching and research university that
attracts outstanding students and is a desired place of
work for top faculty.
The universitys risk appetite statement acknowledges

that risk is present in almost every activity. The critical
question in establishing the risk appetite was How willing
is the university to accept risk related to each area? In

thinking through the process, members of management
used a continuum (Exhibit 4) to express risk appetite for
the universitys major objectives (teaching, research,
service, and operational efficiency). They placed various
risks along the continuum as a basis for discussion at the
highest levels.
Exhibit 4
Acceptable
Not Acceptable
Increased
costs due to
incompatibility
with legacy
computer
systems
Reduced
security of IT
Reduced
teaching
reputation
From an operational viewpoint, for example, management

assigned a high risk appetite to the cost of computer
incompatibility, a more moderate risk appetite to issues
of teaching excellence, a low risk appetite to information
system security, and a very low risk appetite to its
reputation as a leading research organization.
The university found that ordering its risk appetites across
the continuum helped it shape a risk statement. Putting this
into practice, the university

exhibited a higher risk appetite when approving a new

computer system that offered greater processing
capacity but also had potential compatibility issues with
legacy systems;
Reduced
research
reputation
exhibited a low risk appetite for significant breaches of

security or unauthorized access to classified records
(the new system was viewed as better controlled than
the legacy system, thus supporting the decision to
approve the new system);
expressed a moderate risk appetite for teaching

quality; and
expressed a very low risk appetite for risks that would
significantly reduce its research reputation.
www.co s o.o rg
This example illustrates how risk appetite and strategy

interact at the highest levels of an organization. The
discussion of risk appetite guided the universitys
strategies for dealing with issues such as budget cuts and
their effect on teaching, research, service, and operations.
Financial Services Organization: This company
considers quantitative measures to be part of setting risk

appetite, and it focuses on economic capital as a primary
measure. The company manages its financial operations
to attain a reasoned risk/return relationship, which serves
as a guideline for acceptable credit risks, market risks,
and liquidity risks. The companys business operations also
involve risks related to strategic, reporting, compliance,
and operations objectives.
www.co s o.o rg
This organizations view of risk appetite specifies not only

risk appetite but also acceptable tolerances around that
risk appetite that require action to be taken. For example,
the company communicates its risk appetite for loan
impairment losses by stating that such losses should not
exceed 0.25% of the loan portfolio. The company has a
low tolerance for exceeding this level, and significant
remediation is expected should losses go beyond 0.28%.
The same company has a low risk appetite related to its
insurance business, stating that claims incurred should be
no more than 70% of insurance premium revenue.
This organization reviews its risk appetite annually,
adjusting it by type of risk and setting target values for
risk-specific indicators in light of the economic cycle and
market prospects. The board reviews the risk appetite and
associated policies whenever the economic outlook
changes significantly.
Risk Appetite and Risk Tolerance

Risk tolerance relates to risk appetite but differs in one
fundamental way: risk tolerance represents the application of
risk appetite to specific objectives. Risk tolerance is defined as:
The acceptable level of variation relative to achievement
of a specific objective, and often is best measured in the
same units as those used to measure the related objective.
In setting risk tolerance, management considers the
relative importance of the related objective and aligns
risk tolerances with risk appetite. Operating within risk
tolerances helps ensure that the entity remains within
its risk appetite and, in turn, that the entity will achieve
its objectives.4
While risk appetite is broad, risk tolerance is tactical and
operational. Risk tolerance must be expressed in such a way
that it can be
mapped into the same metrics the organization uses to
measure success;
applied to all four categories of objectives (strategic,
operations, reporting, and compliance); and
implemented by operational personnel throughout
the organization.
Because risk tolerance is defined within the context of
objectives and risk appetite, it should be communicated
using the metrics in place to measure performance. In that
way, risk tolerance sets the boundaries of acceptable
Risk tolerances guide operating units as they implement risk

appetite within their sphere of operation. Risk tolerances
communicate a degree of flexibility, while risk appetite sets
a limit beyond which additional risk should not be taken.
performance variability. A simple example in the financial

industry would be to state an appetite for risks associated
with collateralized debt obligations (CDO) where the CDOs
are divided into tranches reflecting the estimated credit
worthiness of the underlying debt. An entity buying these
CDOs may set minimum risk rating levels for these tranches
and then set a tolerance reflecting the maximum downside
risk that is acceptable.
Some tolerances are easy to express in qualitative terms.
For example, an organization may have a low risk appetite
for non-compliance with laws and regulations and may
communicate a similarly low tolerance for violations for
example, a zero tolerance for some types of violations
and slightly higher tolerances for other types of violations.
Or tolerance may be stated in quantitative terms. A company
could say that it requires backup on its computer systems so
that the likelihood of computer failure is less than 0.01%.
Risk tolerances are always related to risk appetite and
objectives (Exhibit 5). Tolerances can apply to detailed
areas such as compliance, computer security, product
quality, or interest rate variability. Risk appetite and
risk tolerances, together with objectives, guide the
organizations actions.
Exhibit 5
Management
sets
OBJECTIVES
with board oversight.

Management sets
TOLERANCES
Management, with board

review and concurrence,
articulates a
RISK APPETITE
that is acceptable in pursuit
of those objectives.
around risks acceptable at the

organizational unit level
or functional unit
level in measuring the
achievement of objectives.
COSO, Enterprise Risk Management Integrated Framework, p. 20.
www.co s o.o rg
Most organizations have multiple operational objectives

related to profitability, some of which might create additional
or complementary risks. For example, the managers of an
aerospace company might want to improve a products
profitability but know the company has a low risk appetite
for not meeting client expectations. They know they cannot
reduce product costs if such changes would decrease
performance. For example, the company might use new
technology, but it cannot use inferior components.
To further illustrate, assume management and the board
have set specific profit objectives by product line for
example, maintain a specific gross margin or return on
capital for the product line. But they have communicated a
low risk appetite for product failure, for loss of customers
because of product quality or delivery, and for potential
lawsuits related to product design or performance. The
articulation of risk tolerances helps guide the companys
operational development.
Linking Risk Appetite and Risk Tolerance
The following examples illustrate the relationship between

risk appetite and related risk tolerances.
Aerospace Supplier: This company translates its
risk appetite statement into tolerances for operational

implementation. A high-level objective is to grow by 8%
a year (revenue and operating earnings) by working with
customers to improve products and market share. Because
of the long-term nature of its supply arrangements and
product development, the company has communicated the
broad parameters of its risk appetite, which then cascade
into risk tolerances relating to operations, reporting, and
compliance, as shown below. While the company seeks to
grow at this rate, acquisitions should not put the companys
capital structure at risk. There is a low risk appetite for
allowing the capital structure to be so leveraged that it
hinders the companys future flexibility or ability to make
strategic acquisitions.
www.co s o.o rg
Operations Tolerances
Near zero risk tolerance for product defects
Low risk tolerance for sourcing products that fail to
meet the companys quality standards
Low, but not zero, risk tolerance for meeting customer
orders on time, and a very low tolerance for failing to
meet demands within x number of days
High risk tolerance for potential failure in pursuing
research that will enable the companys product to
better control, and increase the efficiency of, energy use
Reporting Tolerances
Low risk tolerance concerning the quality, timing, and
accessibility of data needed to run the business
Very low risk tolerance concerning the possibility of
significant or material deficiencies in internal control
A low risk tolerance related to financial reporting quality
(timeliness, transparency, GAAP, etc.)
Compliance Tolerances
Near zero risk tolerance for violations of regulatory
requirements or the companys code of ethics
Company management has been comfortable communicating

risk appetite through its actions and performance reviews.
However, as the company has grown, it has found that the
risk appetite is not fully understood, especially among new
operational units. Nor is it understood that policies relate
to objectives and are often designed to minimize the risks
involved in pursuing those objectives. One division, for
instance, failed to follow a company policy because it did
not fully understand that the policy was in place to mitigate a
significant risk, thus leading to losses. Linking the policy to the
risk and risk appetite would have led to better mitigation of the
underlying risks.
University: The university in our earlier example has a very
low appetite for risk associated with its research reputation.

However, given budget shortages, the university also knows it
cannot make the same commitment to research and teaching
as in the past. The organization has expressed a higher risk
appetite for actions resulting in lower-quality teaching. In
other words, research that leads to better understanding and
innovation is extremely important, but the quality of teaching,
though important, is an area where the university can accept
more risk for potential decreases.
The university communicated its risk appetite in broad
terms, both through the university and, as a public institution,
within the state. However, to operationalize the risk appetite
within each of its schools, the university had to express
risk tolerances for the two key objectives of excellence in
research and teaching while dealing with a 10% budget
decrease. The risk tolerances were expressed as follows.
Research: Tolerance Statements

Consistent With Low Risk Appetite
The university does not expect any decrease in the
nature, quality, or number of publications related to its
research mission.
The university does not expect any decrease in the
number or dollar value of outside research grants
generated by faculty.
Teaching: Tolerance Statements Consistent
With Moderate Risk Appetite
Student teaching evaluations should not decline by
more than 5%.

Where individual schools within the university are

ranked by outside evaluators on student preparedness
and quality of students, there should be no more than
a 5% decline.
The caliber of students wanting to attend the university

should not decline by more than 2%, as measured by
standard university admissions data such as SAT or
ACT scores, percentile ranking in high school
graduating class, or extent of community service
before attending university.
The idea behind the risk tolerances is that if the university falls
below any of the measures, corrective action will take place.
Corrections will come not from adjusting the risk appetite but
from reassessing the risk appetite and the strategies the
university has implemented in the context of the risk appetite.
www.co s o.o rg
14
Examples of Risk Tolerance Statements

The following examples from organizations show how risk tolerance might be stated and aligned with broader risk appetite.
Risk Appetite
Risk Tolerance
The organization has a higher risk appetite related

to strategic objectives and is willing to accept higher
losses in the pursuit of higher returns.
While we expect a return of 18% on this investment,

we are not willing to take more than a 25% chance
that the investment leads to a loss of more than 50%
of our existing capital.
The organization has a low risk appetite related to

risky ventures and, therefore, is willing to invest in new
business but with a low appetite for potential losses.
We will not accept more than a 5% risk that a new

line of business will reduce our operating earnings
by more than 5% over the next ten years.
A health services organization places patient safety

amongst its highest priorities. The organization
also understands the need to balance the level of
immediate response to all patient needs with the cost
of providing such service. The organization has a low
risk appetite related to patient safety but a higher
appetite related to response to all patient needs.
We strive to treat all emergency room patients

within two hours and critically ill patients within
15 minutes. However, management accepts that in
rare situations (5% of the time) patients in need of
non-life-threatening attention may not receive that
attention for up to four hours.
A retail company has a low risk appetite related to the

social and economic costs for sourced products from
foreign locations that could be accused of being child
sweatshops or having unhealthy working conditions.
For purchasing agents, the risk tolerance is set

at near zero for procuring products that do not
meet the organizations quality and sourcing
requirements.
A manufacturer of engineered wood products

operates in a highly competitive market. To compete,
the company has adopted a higher risk appetite
relating to product defects in accepting the cost
savings from lower-quality raw materials.
The company has set a target for production defects

of one flaw per 1,000 board feet. Production staff
may accept defect rates up to 50% above this target
(i.e., 1.5 flaws per 1,000 board feet) if cost savings
from using lower-cost materials is at least 10%.
www.co s o.o rg
15
Developing Risk Appetite

We have identified the characteristics of an effective risk
appetite statement and noted how those characteristics
are useful in managing risk. We have also examined the
relationship between risk appetite and risk tolerances.
Now we will discuss how an organization can bring out the
many implicit feelings that management and the board
may have about what they believe is the organizations
risk appetite and how discussion of those feelings leads to
development of risk appetite.
Develop/
Revise
Risk
Appetite
Monitor
Communicate
Developing a risk appetite is not an end in itself and should

not require an inordinate amount of time. Remember the
purposes of risk appetite are
to provide effective communication throughout the
organization in order to drive the implementation of
enterprise risk management;
to change discussions about risk so that they involve
questioning of whether risks are properly identified and
managed within the risk appetite; and
Developing risk appetite is about managing the organization.

It is not about developing a statement to be filed in a report.
There are many ways to create a clear statement of risk
appetite. Organizations should identify the parameters of their
risk appetite along key strategic, operational, reporting,
and compliance objectives.
to provide a basis for further discussion of risk appetite

as strategies and objectives change.
Also, keep in mind that any expression of risk appetite must
be preceded by a discussion of strategies and objectives.
The risk appetite must be linked to those objectives.
Management and boards often use one of three
approaches to discuss and develop their risk appetite: (1)
facilitated discussions, (2) discussions related to objectives
and strategies, or (3) development of performance models.
Facilitated Discussions
Facilitated discussions can be very effective for a variety
of organizations. After several iterations, management
and the board can develop a risk appetite statement
that reflects the combined views of the organizations
leadership and governance bodies.
The major advantage of this approach is that the
facilitators encourage management and the board to
clearly prioritize their objectives and their risk appetite.
In addition, various scenarios can be discussed to see
how the risk appetite would influence decision making
throughout the organization. When discussing risk
appetite, those involved should keep the organizations
strategic plan, including goals and mission, at the forefront.
A questionnaire can help capture views on risk appetite

and business scenarios. Exhibit 6 shows an example. Note
that the questions are broad and should be tailored to the
unique factors that drive an organizations success.
Discussions Related to Objectives and Strategies
Often the risk appetite an organization is willing to accept
becomes more evident when management considers
major issues facing the organization, such as new product
lines, acquisitions, or joint ventures. Management of
organizations with a lower risk appetite will usually react
differently to acquisition, expansion, competition, and
market volatility than will peers with a higher risk appetite.
Reviewing and assessing these reactions can provide
insight into the organizations current risk appetite.
This approach allows management to go the extra step
in discussing major strategies because it asks what the
perceived risks are in pursuing objectives. The board then
reviews and supports managements identification and
communication of risk appetite as it relates to
specific objectives.
www.co s o.o rg
16
Exhibit 6
Questions to Facilitate Discussion of Risk Appetite at Management and Board Level
1. On a scale of 1 to 10, with 1 being the lowest, describe what you believe the organizations overall risk
appetite has been and what you think it should be. Explain any differences between what you perceive it
has been and what you believe it should be. Relate this to your number one strategic goal.
2. Various operations help an organization achieve its objectives. Using the categories below, or other
categories consistent with the organizations operations, rate the desired risk appetite related to the
following (rating can be broad, such as high, medium, or low, or precise, such as specific metrics that
should not be exceeded):
a. Meeting customer requirements
b. Employee health and safety
c. Environmental responsibility
d. Financial reporting
e. Operational performance
f. Regulatory compliance
g. Shareholder expectations
h. Strategic initiatives / growth targets

As you rate each category, indicate areas where you believe the organization is taking either too much or
too little risk in pursuing its objectives.
3. How would you rate the effectiveness of the organizations process for identifying, assessing, managing,
and reporting risks in relation to the overall risk appetite? What are the major areas for improvement?
4. Are managements strategies communicated sufficiently for there to be meaningful discussion of risk
appetite in pursuit of those strategies, both at the broad organizational level and at the operational level,
and for consistency to be analyzed?
5. How satisfied are you that the board is providing effective oversight of the risk appetite through its
governance process? This includes board committees and/or the board itself to help set the appetite and
to monitor over time that management is adhering to the overall risk appetite in pursuit of value.
6. Whom do you see as more accepting of risk, or more willing to take risks to meet the goals of the organization?
a. Management
b. Board
c. Management and board have similar levels of acceptable risk
7. Does the organization motivate management (senior management and operational management) to take higher
than desired risks because of the compensation plans in place? If yes, how do you believe the compensation plans
should be modified to bring approaches for generating high performance within the risk appetite?
8. What do you believe the organization should do?
a. Reduce its risk appetite
b. Increase its risk appetite
c. Make no change
9. Do you believe there are risks considered to be above the organizations existing risk appetite that need to
be reduced? In other words, are there areas where the risk appetite, as currently used, is too low?
10. What risks over the past five years were, in your view, above the organizations risk appetite? Were the risks
understood when a strategy was developed? How could management have communicated its risk appetite
so that the board could both (a) evaluate the risk appetite and (b) provide proper oversight? How could
management have communicated its risk appetite so as to hold operational units to actions consistent with
the risk appetite?
www.co s o.o rg
One advantage to this approach is that the board can be seen

as supporting or challenging managements risk appetite.
Another is that management gains a sense of the boards
risk appetite for specific strategies and can incorporate
that knowledge into a risk management process. The
major disadvantage of this approach is that it can be less
comprehensive. It often does not generate the specificity
needed for the organizations day-to-day activities.
Development of Performance Models
Some organizations, particularly financial institutions, use
quantitative measures to express their overall risk appetite.
They often arrive at these measures through performance
modelling.
A company could, for instance, use economic capital to
express risk appetite. Economic capital is the amount of
capital a financial institution needs to remain solvent. This
determination is based both on regulatory requirements and
on managements assessment of how much economic
capital the institution needs to retain.
17
As part of developing (and monitoring) risk appetite, a

company may model its overall risk profile. This involves
taking bottom-up risk information and developing models
that consider company-specific risks, including industry
factors and broad economic factors, to create a calculated
risk profile. The profile can then be compared to the overall
risk appetite, helping management and the board to discuss
how much risk the organization is prepared to accept. Some
organizations also review key ratios from peer companies
and industries to gain more input into the risk level suitable
for their organization.
Modelling is typically only one part of the process of
setting risk appetite. For one thing, an organization needs
considerable data to prepare these calculations. For
another, there are usually certain risks that are difficult to
quantify and model with precision. Management and the
board still need to debate and discuss the levels above which
capital at risk is seen to be too high and in excess of appetite.
As an example, management might set its economic capital

at 6% of total assets. As the organization models different
scenarios of economic activity, economic situations, and its
asset portfolio, it needs to set some probability around the
ability to maintain economic capital. A management
and board with a low risk appetite might want to be 99.9%
confident (999 out of 1,000 model results) that economic
activities will not place the institution below its desired level
of economic capital. A company with a higher risk appetite
might start with the same dollar amount but require a
confidence level of only 95% (950 out of 1,000 model results).
Thus, risk appetite can be composed of both dollar elements
and probability elements.
www.co s o.o rg
18

Once an overall risk appetite is developed, management
must then choose the right mechanism for communicating
it. As we noted earlier, risk appetite statements will vary,
and organizations may communicate risk appetite at
various levels of detail or precision. The point is that each
organization should determine the best way to communicate
risk appetite to operational leaders in a specific enough
manner that the organization can monitor whether risks are
being managed within that appetite.
Develop/
Revise
Risk
Appetite
Monitor
Communicate
To be effective, risk appetite must be

operationalized through appropriate risk tolerances;
www.co s o.o rg
1
Likely
Unlikely
Almost certain
Insignicant 4
Catastrophic
High
Major
Risk
Moderate
Appetite
Minor
2
Likely
Insignicant 4
Almost certain
The advantage of this approach is that it is simple to convey

the level above which risks are seen as unacceptable. We
also find that discussions with management and the board on
the relative positioning of the bands can draw out important
differences between managements and the boards views on
desired risk appetite.
Catastrophic
Low
Major
Risk
Moderate
Appetite
Minor
Possible
Some organizations use graphics, like those at right, in

discussing risk appetite. A common approach is to apply
some form of color banding within a heat map that indicates
acceptable versus unacceptable risk levels. With this
approach, risks are grouped by objective, summarized, and
then plotted on the risk map. The organization sets either the
assessment criteria or the location of the color banding to
express higher versus lower risk appetites. For instance, the
heat maps on the right show that risks related to objectives 1
and 2 would exceed the appetite of a company with a low risk
appetite, but not necessarily that of a company with a high
risk appetite. Risks related to objective 3 would exceed the
appetite of both companies.
Unlikely
Broad Risk Appetite Statement

Organizations that communicate overall risk appetite in
broad terms may develop high-level statements that reflect
acceptable risk levels in pursuing their objectives.
The Organization operates within a low overall risk

range. The Organizations lowest risk appetite relates to
safety and compliance objectives, including employee
health and safety, with a marginally higher risk appetite
towards its strategic, reporting, and operations
objectives. This means that reducing to reasonably
practicable levels the risks originating from various
medical systems, products, equipment, and our work
environment, and meeting our legal obligations will take
priority over other business objectives.
Possible
We have encountered three main approaches for

communicating risk appetite: (1) expressing overall risk
appetite using broad statements, (2) expressing risk appetite
for each major class of organizational objectives, and (3)
expressing risk appetite for different categories of risk.
Risks Related to Organizational Objectives

Organizations that communicate risk appetite for each major
class of organizational objectives are likely to communicate
risk appetite in some form of statement. Consider the risk
appetite statement from the health care organization we
referred to earlier:
Almost never
specific enough to be monitored by management and

others responsible for risk management.
The broad descriptions are effective when they are partitioned

to show that not all objectives have the same risk appetite.
Almost never
stated in a way that assists management in decision

making; and
The advantage of communicating risk appetite according

to categories of risk is that management can exercise
judgment about acceptable levels given the unique
considerations of each group of risks. By allowing for
greater judgment, this approach reduces the perception
that risk management is overly prescriptive.
ce
ian
ng
Co
pl
rti
po
Re
Op
er
at
io
ic
ns
Risk appetite and risk tolerances are set across the

organization. Risk appetite is set at the highest level of the
organization in conjunction with goals and objectives. As
risk appetite and objectives are communicated throughout
the organization (subsidiary, division, or business unit level)
the strategic goals and risk appetite are expressed in more
specific performance terms. Strategies are reflected in
performance objectives, and risk appetite is expressed
in terms of risk tolerance. The more precise articulation
of performance objectives and risk tolerances helps
management to identify situations where corrective actions
are needed. Performance metrics and risk tolerances that
are more specific lend themselves to better monitoring.
Internal Environment
Objective Setting
Event Identification
Risk Assessment
Risk Response
Control Activities
Subsidiary
Business Unit
Division
Entity-Level
A mining company we are aware of has specific objectives

for cash flow and capital structure that include maintaining
low volatility of cash flow. There are many causes of
cash flow volatility, ranging from operations to uncertain
commodity prices. Management believes that investors
understand commodity price risk, and it has pursued
objectives that enable the company to benefit from price
increases while being exposed to losses from price
decreases. Management believes that this price risk
even though it can result in volatile earnings is within
the appetite of the organization (and its stakeholders).
Therefore, the company has not attempted to mitigate
this exposure through a commodity price hedge program.
Conversely, the same company is unwilling to accept a
similar level of cash flow volatility caused by production
delays, and it has adopted rigorous processes to maintain
steady production.
Risk appetite needs to be communicated by management,

embraced by the board, and then integrated across the
organization. The ERM framework is often depicted as a
cube (see below). It is important not to overlook the side of
the cube, which shows that all units must understand the
organizations risk appetite and related risk tolerances.
ra
te
g
Categories of Risk
The third option is to communicate appetite for categories
of risk. Some organizations use broad, generic risk
categories, such as economic, environmental, political,
personnel, or technology, in their risk appetite statements.
Others use more tailored risk categories that apply to their
field. For example, a company in information processing
may group risks related to system availability, data security
and privacy, system scalability, system design, and
release management.
Risk Appetite Cascades Through the

Organization
The method of communicating a risk appetite statement
is important, but so is the ability to communicate that
statement across the organization in a way that ensures
operations are consistent with the risk appetite. It is
especially important for those who pursue the operational
tactics related to organizational objectives (e.g., local
sales forces, country managers, strategic business units)
to clearly understand and be aligned with risk appetite.
All too often, the risk appetite and tolerances set by the
organization are not adhered to or understood in context by
those managing the day-to-day business, facing customers
and potential risks every day.
St
The advantage of this approach is that it allows for more

delineation between the levels of acceptable risk for each
class of objectives. It does not, for instance, treat risks
related to legal compliance the same way as risks related
to operations. This approach may also help with decision
making, especially if resources are limited and need to be
allocated across a companys organizational units. Another
advantage is that viewing risks in relation to classes of
objectives requires less effort than, say, the third approach
below. The challenge is to develop a statement that
accommodates specific risk types that should be viewed
differently in terms of acceptable level of risk.
19
Information & Communication

Monitoring
www.co s o.o rg
20
Monitoring and Updating Risk Appetite

Once an organizations risk appetite is developed and
communicated, management, with board support, must
revisit and reinforce it. Risk appetite cannot be set once and
then left alone for extended periods. Rather, it should be
reviewed and incorporated into decisions about how the
organization operates. This is especially important if the
organizations business model begins to change.
Management cannot just assume that responsible
individuals will implement risk management within the
appropriate risk appetite. Therefore, some organizations will
review the application of risk appetite through a series of
monitoring activities. Management should monitor the
organizations activities for consistency with risk appetite
through the specifics identified with risk tolerances. Most
organizations have key performance risk metrics that they
use to measure performance. It is easy to integrate risk
tolerances into the monitoring process used to evaluate
performance. Internal auditing can provide independent
insight on the effectiveness of such processes.
Creating a Culture
For many organizations, monitoring risk tolerances requires a
culture that is aware of risk and risk appetite. Management,
by revisiting and reinforcing risk appetite, is in a position to
create a culture whose organizational goals are consistent
with the boards, and to hold those responsible for implementing
risk management within the risk appetite parameters.
Many organizations are effective at creating a risk-aware
culture: a culture that emanates from senior management,
cascades through the organization, and is supported by
the board. In an effective culture, each member of the
organization has a clear idea of what is acceptable, whether
in relation to behaving ethically, pursuing the wrong objectives,
or encountering too much risk in pursuing the right objectives.
www.co s o.o rg
Develop/
Revise
Risk
Appetite
Monitor
Communicate
Creating a culture is one way of reinforcing overall risk

appetite. The approach is best used when the organization
has a well-communicated risk appetite and associated risk
tolerances, to the point at which the following outcomes exist:
Consistent implementation across units
Effective monitoring and communication of risk and
changes in risk appetite
Consistent understanding of risk appetite and related
tolerances for each organizational unit
Consistency between risk appetite, objectives, and
relevant reward systems
This approach draws on ongoing and separate evaluations
conducted as part of the organizations monitoring. The
individuals doing the monitoring consider whether the
objectives being set and the risk response decisions being
made are consistent with the organizations stated risk
appetite. Any variation from the stated (or desired) risk
appetite is then reported to management and the board as
part of the normal internal reporting process.
21
Roles
It is managements role to develop the risk appetite and
to obtain the boards agreement that the risk appetite is
suitable for the organization. We believe that the board
is in place to oversee management and to monitor the
broader risk management process, including whether the
organization is adhering to its stated risk appetite. Any
board, serving any organization of any size or structure (forprofit, not-for-profit, private), has a fiduciary responsibility to
question managements development and implementation of
a risk appetite and to require changes if it believes the risk
appetite is either badly communicated or inconsistent with
shareholder values.
Board Oversight
Management
Develop/
Revise
Effective board oversight of an organizations risk appetite

should include
clear discussion of the organizations objectives and
risk appetite;
Risk
Appetite
Monitor
Communicate
oversight of the organizations compensation plan for

consistency with risk appetite;
oversight of managements risk identification when
pursuing strategies to determine whether the risks
exceed the risk appetite;

oversight of strategies and objectives to determine

whether the pursuit of some objectives may create
unintended consequences or organizational risks in
other areas; and
a governance structure that requires regular

conversations on risk appetite, through the board and
board committees, concerning matters such as
strategy formulation and execution, M&A activity, and
business cases to pursue major new initiatives.
Boards are very good at questioning strategies. They are only

a step away from addressing meaningful questions that can
help with setting the organizations risk appetite. For example,
when the board asks how much an organization should pay
for an acquisition, it is an expression of risk appetite.
Governance does not stop with board oversight. It includes

managements development of the infrastructure for risk
management and the allocation of resources across the
organization. Exhibit 7 is a summary of matters for the board
and management to consider in evaluating how effective
their processes are for developing, communicating, and
monitoring risk appetite.
www.co s o.o rg
22
Exhibit 7
Board and Management Responsibilities
1. Management establishes risk appetite: An organization cannot know how well it is managing risk unless it
establishes ranges of acceptable risk it can take in pursuit of its objectives. In doing so, management must
effectively and clearly communicate:
a. Goals and objectives
b. Strategies
c. Metrics (to know whether objectives are being achieved)
d. Relevant time periods for pursuing the objectives
e. Ranges of risk the organization is willing to take in pursuing the objectives
2. Board oversees risk appetite: Oversight of the risk appetite (or acceptable ranges of acceptable risk)
should be considered at the board level in conjunction with the senior management team.
3. Applies throughout organization: Risk appetite needs to be applied regularly throughout all functional
units of the organization. Culture is important: the organization must work to build the boards view of risk
appetite into the organizational culture.
4. Aligns with stakeholders and managers: Because individuals are accountable for their results, every
organization needs a robust governance process to ensure that compensation and incentive systems are
aligned with the organizations objectives and are managed to fall within the organizations risk appetite.
5. Manages risks and risk appetite over time: Organizations need to understand that risk appetites
may change over time. Boards must be proactive on two levels:
a. Communicating their articulation of risk appetite
b. Monitoring organizational actions, processes, etc., to determine whether organizational activity has
strayed outside the organizations risk appetite
6. Monitors to ensure adherence to risk appetite: Adherence to an organizations risk appetite, as well as to
its risk management processes, should be monitored regularly. The results of the monitoring should be
reported to the audit committee and/or board and to the relevant members of executive management.
7. Supports culture: The tone at the top influences the culture of the organization. The tone can be either
positive or negative in ensuring that risks are managed within acceptable limits. Ideally, prudent risk taking
is built into the organizations culture in its public statement of core values.
8. Considers resources: It takes effort to operate within the organizations risk appetite. Resources must be
available and dedicated to operating within this appetite.
9. Communicates through strategies and objectives: Risk appetite is communicated effectively only if the
organization can clearly communicate its major strategies and objectives at both the global level and the
functional/operational level.
10. Clearly communicates how much risk the organization is willing to accept at all levels: Risk appetite and
risk tolerance are complementary concepts. They can be combined to determine acceptable ranges of risk
for the organization.
Risk appetite is developed by management and reviewed by the board. COSOs Enterprise Risk Management Integrated Framework
emphasizes the boards important role in overseeing risk management. Oversight should begin with a studied discussion
and review of managements articulation of risk appetite relative to the organizations strategies.
www.co s o.o rg
23
Summary of Considerations
The COSO Enterprise Risk Management Integrated
Framework sets out five principles related to risk appetite:
1. It is a guidepost in strategy setting.
2. It guides resource allocation.
3. It aligns organization, people, processes, and
infrastructure.
4. It reflects the entitys risk management philosophy
and influences the culture and operating style.
5. It is considered in strategy setting so that strategy
aligns with risk appetite.
Risk appetite does not exist in a vacuum; rather, it is an
integral part of an organizations strategies for achieving
objectives. The concept of risk appetite permeates all
organizations, from charities and governments to small
businesses and publicly traded corporations.
A statement of risk appetite is an effective way to communicate

across an organization a sense of acceptable risks. In addition,
it provides a basis for evaluating and monitoring the amount of
risk an organization faces to determine whether the risk has
risen above an acceptable range.
Organizations can, and should, come to terms with what
they believe to be their appetite for risk. Once stated, risk
appetite can be communicated and refined over time as the
organization becomes more experienced with the concept.
Most importantly, developing risk appetite is the start of
an organizations commitment to effective enterprise risk
management. As with pursuing corporate objectives, the
end objective is adding value through effective enterprise
risk management in pursuit of organizational goals.
Developing and communicating a risk appetite moves
organizations in that direction.
www.co s o.o rg
About COSO
Originally formed in 1985, COSO is a joint initiative of five private sector organizations and is dedicated to providing thought
leadership through the development of frameworks and guidance on enterprise risk management (ERM), internal control
and fraud deterrence. COSOs supporting organizations are The Institute of internal Auditors (IIA), the American Accounting
Association (AAA), the American Institute of Certified Public Accountants (AICPA), Financial Executives International (FEI),
and the Institute of Management Accountants (IMA).
About the Authors

Dr. Larry Rittenberg is the Ernst & Young Professor of Accounting at the University of Wisconsin-Madison School of
Business. He is one of only eight academics on the list of the United States 100 most influential people in finance. Dr. Rittenberg
was on the COSO steering committee that oversaw the development of Enterprise Risk Management Integrated Framework
and later served as chair of COSO. As chair, he led the effort to provide guidance for small and midsize companies on
developing effective internal controls, and later led COSO in developing guidance on monitoring of internal controls.
On the University of Wisconsin faculty since 1976, Dr. Rittenberg teaches in the area of audit and assurance, including risk
management and corporate governance. His current research deals with the effectiveness of audit committees, corporate
governance, and assurance services. He has received The Institute of Internal Auditors highest award, the Bradford
Cadmus Memorial Award, for his contributions to the internal auditing profession.
Frank Martens is a Director in the Advisory Practice of PricewaterhouseCoopers (PwC). He provides services related to
enterprise risk management, internal audit, and internal control to a wide range of companies. Mr. Martens is a Chartered
Accountant with over 20 years of external audit experience.
Mr. Martens was one of the principal contributors from PwC in developing COSOs Enterprise Risk Management
Integrated Framework. He was also a principal contributor to COSOs Internal Control over Financial Reporting Guidance
for Smaller Public Companies, a guidance document for using COSOs Internal Control Integrated Framework.
Note to Readers
The information contained herein is of a general nature and based on authorities that are subject to change. Applicability of
the information to specific situations should bedetermined through consultation with your professional adviser. This thought
paper represents the views of the authors only, and does not necessarily represent the views or professional advice of the
University of Wisconsin, PwC, or COSO.
www.co s o.o rg
C o m m i t t e e
o f
S p o n s o r i n g
o f
t h e
T r e a d w a y
C o m m i s s i o n
R I S K
A S S E S S M E N T
I N
P R A C T I C E
By
Deloitte & Touche LLP
Dr. Patchin Curtis | Mark Carey
The information contained herein is of a general nature and based on authorities that are subject to change. Applicability of the information to
specific situations should be determined through consultation with your professional adviser, and this paper should not be considered substitute
for the services of such advisors, nor should it be used as a basis for any decision or action that may affect your organization.
Authors
Principal Contributors
Dr. Patchin Curtis

Director,
Mark Carey
Partner,
COSO Board Members

David L. Landsittel
COSO Chair
Marie N. Hollein
Douglas F. Prawitt
Chuck E. Landes
Richard F. Chambers
Sandra Richtermeyer
Preface
This project was commissioned by the Committee of Sponsoring Organizations of the Treadway Commission
(COSO), which is dedicated to providing thought leadership through the development of comprehensive
frameworks and guidance on enterprise risk management, internal control, and fraud deterrence designed to
improve organizational performance and governance and to reduce the extent of fraud in organizations.
COSO is a private-sector initiative jointly sponsored and funded by the following organizations:

The Institute of Management Accountants (IMA)

www.co s o.o rg
October 2012
1234567890 PIP 198765432
American Institute of Certified Public Accountants licensing and permissions agent for COSO copyrighted materials.
Direct all inquiries to copyright@aicpa.org or AICPA, Attn: Manager, Rights and Permissions, 220 Leigh Farm Rd.,
www.co s o.o rg
Thought Leadership in ERM | Risk Assessment in Practice | iii
Contents
Page
Introduction
The Risk Assessment Process
Develop Assessment Criteria
Assess Risks
Assess Risk Interactions
12
Prioritize Risks
14
Putting It into Practice
18
About COSO
19
About the Authors
19
www.co s o.o rg
Thought Leadership in ERM | Risk Assessment in Practice |
Introduction
Value is a function of risk and return. Every decision
either increases, preserves, or erodes value. Given that
risk is integral to the pursuit of value, strategic-minded
enterprises do not strive to eliminate risk or even to
minimize it, a perspective that represents a critical change
from the traditional view of risk as something to avoid.
Rather, these enterprises seek to manage risk exposures
across all parts of their organizations so that, at any given
time, they incur just enough of the right kinds of riskno
more, no lessto effectively pursue strategic goals. This is
the sweet spot, or optimal risk-taking zone, referred to in
exhibit 1.
Thats why risk assessment is important. Its the way in
which enterprises get a handle on how significant each
risk is to the achievement of their overall goals.
To accomplish this, enterprises require a risk assessment

process that is practical, sustainable, and easy to
understand. The process must proceed in a structured
and disciplined fashion. It must be correctly sized to the
enterprises size, complexity, and geographic reach. While
enterprise-wide risk management (ERM) is a relatively new
discipline,1 application techniques have been evolving
over the last decade. The purpose of this paper is to
provide leadership with an overview of risk assessment
approaches and techniques that have emerged as the most
useful and sustainable for decision-making. It represents
another in a series of papers published by Committee of
Sponsoring Organizations of the Treadway Commission
(COSO) aimed at helping organizations move up the
maturity curve in their ongoing development of a robust
ERM process.
Exhibit 1: Optimal Risk-Taking
Insufficient
Risk-Taking
Optimal
Risk-Taking
Excessive
Risk-Taking
Expected
Enterprise
Value
Sweet Spot
Risk Level
Committee of Sponsoring Organizations of the Treadway Commission (COSO) Enterprise Risk Management Integrated
Framework, 2004.
1
www.co s o.o rg
2 | Risk Assessment in Practice | Thought Leadership in ERM
The Risk Assessment Process

Within the COSO ERM framework,2 risk assessment follows
event identification and precedes risk response. Its purpose
is to assess how big the risks are, both individually and
collectively, in order to focus managements attention on
the most important threats and opportunities, and to lay
the groundwork for risk response. Risk assessment is all
about measuring and prioritizing risks so that risk levels are
managed within defined tolerance thresholds without being
overcontrolled or forgoing desirable opportunities.
Events that may trigger risk assessment include the initial

establishment of an ERM program, a periodic refresh, the
start of a new project, a merger, acquisition, or divestiture,
or a major restructuring. Some risks are dynamic and
require continual ongoing monitoring and assessment, such
as certain market and production risks. Other risks are more
static and require reassessment on a periodic basis with
ongoing monitoring triggering an alert to reassess sooner
should circumstances change.
Exhibit 2: Assess Risks Process Flow Diagram

Assess Risks
Identify
Risks
Develop
Assessment
Criteria
Assess
Risks
Identify risks. The risk (or event) identification process

precedes risk assessment and produces a comprehensive
list of risks (and often opportunities as well), organized
by risk category (financial, operational, strategic,
compliance) and sub-category (market, credit, liquidity,
etc.) for business units, corporate functions, and capital
projects. At this stage, a wide net is cast to understand the
universe of risks making up the enterprises risk profile.
While each risk captured may be important to management
at the function and business unit level, the list requires
prioritization to focus senior management and board
attention on key risks. This prioritization is accomplished
by performing the risk assessment.
Develop assessment criteria. The first activity within the
risk assessment process is to develop a common set of
assessment criteria to be deployed across business units,
corporate functions, and large capital projects. Risks and
opportunities are typically assessed in terms of impact
and likelihood. Many enterprises recognize the utility
of evaluating risk along additional dimensions such as
vulnerability and speed of onset.
Assess risks. Assessing risks consists of assigning values
to each risk and opportunity using the defined criteria.
This may be accomplished in two stages where an initial
screening of the risks is performed using qualitative
techniques followed by a more quantitative analysis of the
most important risks.
Assess Risk
Interactions
www.co s o.o rg
Respond
to Risks
Assess risk interactions. Risks do not exist in isolation.

Enterprises have come to recognize the importance of
managing risk interactions. Even seemingly insignificant
risks on their own have the potential, as they interact with
other events and conditions, to cause great damage or
create significant opportunity. Therefore, enterprises are
gravitating toward an integrated or holistic view of risks
using techniques such as risk interaction matrices, bow-tie
diagrams, and aggregated probability distributions.
Prioritize risks. Risk prioritization is the process of
determining risk management priorities by comparing the
level of risk against predetermined target risk levels and
tolerance thresholds. Risk is viewed not just in terms of
financial impact and probability, but also subjective criteria
such as health and safety impact, reputational impact,
vulnerability, and speed of onset.
Respond to risks. The results of the risk assessment process
then serve as the primary input to risk responses whereby
response options are examined (accept, reduce, share, or
avoid), cost-benefit analyses performed, a response strategy
formulated, and risk response plans developed.
Discussions of event identification and risk response are
beyond the scope of this paper. For detailed treatment, refer
to the COSO Enterprise Risk Management Integrated
Framework (2004).
COSO, Enterprise Risk Management Integrated Framework (2004).
Prioritize
Risks
Develop Assessment Criteria

Traditional risk analysis defines risk as a function of
likelihood and impact. Indeed, these are important
measures. However, unlikely events occur all too often,
and many likely events dont come to pass. Worse, unlikely
events often occur with astonishing speed. Likelihood and
impact alone do not paint the whole picture.
To answer questions like how fast could the risk arise,
how fast could you respond or recover, and how
much downtime could you tolerate, you need to gauge
vulnerability and speed of onset. By gauging how
vulnerable you are to an event, you develop a picture of
your needs. By gauging how quickly it could happen, you
understand the need for agility and rapid adaptation.
dispersion than three point scales. Ten point scales imply

precision typically unwarranted in qualitative analysis,
and assessors may waste time trying to differentiate
between a rating of six or seven when the difference is
inconsequential and indefensible.
Illustrative scales are provided for impact, likelihood,
vulnerability, and speed of onset. Every enterprise is different
and the scales should be customized to fit the industry, size,
complexity, and culture of the organization in question.
Developing Assessment Scales

Some form of measurement of risk is necessary. Without
a standard of comparison, its simply not possible to
compare and aggregate risks across the organization.
Most organizations define scales for rating risks in terms
of impact, likelihood, and other dimensions. These scales
comprise rating levels and definitions that foster consistent
interpretation and application by different constituencies.
The more descriptive the scales, the more consistent their
interpretation will be by users. The trick is to find the right
balance between simplicity and comprehensiveness.
Impact
Impact (or consequence) refers to the extent to which a
risk event might affect the enterprise. Impact assessment
criteria may include financial, reputational, regulatory,
health, safety, security, environmental, employee,
customer, and operational impacts. Enterprises typically
define impact using a combination of these types of impact
considerations (as illustrated below), given that certain
risks may impact the enterprise financially while other
risks may have a greater impact to reputation or health and
safety. When assigning an impact rating to a risk, assign
the rating for the highest consequence anticipated. For
example, if any one of the criteria for a rating of 5 is met,
then the impact rating assigned is 5 even though other
criteria may fall lower in the scale.
Scales should allow meaningful differentiation for ranking

and prioritization purposes. Five point scales yield better
Some entities define impact scales for opportunities as

well as risks.
www.co s o.o rg
Illustrative Impact Scale

Rating
Descriptor
Definition
5
Extreme
Financial loss of $X million or more3

International long-term negative media coverage; game-changing loss of
market share

Significant prosecution and fines, litigation including class actions,
incarceration of leadership

Significant injuries or fatalities to employees or third parties, such as
customers or vendors

Multiple senior leaders leave
4
Major
Financial loss of $X million up to $X million

National long-term negative media coverage; significant loss of market share

Report to regulator requiring major project for corrective action

Limited in-patient care required for employees or third parties, such as
customers or vendors

Some senior managers leave, high turnover of experienced staff, not
perceived as employer of choice
3
Moderate

National short-term negative media coverage

Report of breach to regulator with immediate correction to be implemented

Out-patient medical treatment required for employees or third parties, such
as customers or vendors

Widespread staff morale problems and high turnover
2
Minor


Local reputational damage
Reportable incident to regulator, no follow up
No or minor injuries to employees or third parties, such as customers or vendors
General staff morale problems and increase in turnover
1
Incidental

Financial loss up to $X million

Local media attention quickly remedied
Not reportable to regulator
No injuries to employees or third parties, such as customers or vendors
Isolated staff dissatisfaction
Financial impact is typically measured in terms of loss or gain, profitability or earnings, or capital.
www.co s o.o rg
Likelihood
Likelihood represents the possibility that a given event
will occur. Likelihood can be expressed using qualitative
terms (frequent, likely, possible, unlikely, rare), as a percent
probability, or as a frequency. When using numerical values,
whether a percentage or frequency, the relevant time period
should be specified such as annual frequency or the more
relative probability over the life of the project or asset.

Sometimes enterprises describe likelihood in more personal
and qualitative terms such as event expected to occur
several times over the course of a career or event not
expected to occur over the course of a career.
Illustrative Likelihood Scale

Rating

Annual Frequency
Descriptor
Definition
Probability
Descriptor
Definition
5
Frequent

Up to once in 2 years
or more
Almost
certain
90% or greater chance of

occurrence over life of asset or project
4
Likely

Once in 2 years up to
once in 25 years
Likely

65% up to 90% chance of occurrence

over life of asset or project
3
Possible

Once in 25 years up to
once in 50 years
Possible


2
Unlikely

Once in 50 years up
to once in 100 years
Unlikely


Once in 100 years or less
Rare

<10% chance of occurrence over life

of asset or project
Rare
www.co s o.o rg
Vulnerability
Vulnerability refers to the susceptibility of the entity to a risk
event in terms of criteria related to the entitys preparedness,
agility, and adaptability. Vulnerability is related to impact
and likelihood. The more vulnerable the entity is to the risk,
the higher the impact will be should the event occur. If risk
responses including controls are not in place and operating
as designed, then the likelihood of an event increases.
Assessing vulnerability allows entities to gauge how well
theyre managing risks.
Vulnerability assessment criteria may include capabilities

to anticipate events such as scenario planning, real options,4
capabilities to prevent events such as risk responses in
place, capabilities to respond and adapt quickly as events
unfold, and capabilities to withstand the event such as
capital buffer and financial strength. Other factors can also
be considered such as the rate of change in the industry or
organization. There is no one-size-fits-all assessment scale.
Every entity must define scales to meet its needs.
Illustrative Vulnerability Scale

Rating
Descriptor
Definition
5
Very High

No scenario planning performed

Lack of enterprise level/process level capabilities to address risks
Responses not implemented
No contingency or crisis management plans in place
4
High

Scenario planning for key strategic risks performed

Low enterprise level/process level capabilities to address risks
Responses partially implemented or not achieving control objectives
Some contingency or crisis management plans in place
3
Medium

Stress testing and sensitivity analysis of scenarios performed

Medium enterprise level/process level capabilities to address risks
Responses implemented and achieving objectives most of the time
Most contingency and crisis management plans in place, limited rehearsals
2
Low
Strategic options defined

Medium to high enterprise level/process level capabilities to address risks

Responses implemented and achieving objectives except under
extreme conditions

Contingency and crisis management plans in place, some rehearsals
1
Very Low

Real options deployed to maximize strategic flexibility

High enterprise level/process level capabilities to address risks
Redundant response mechanisms in place and regularly tested for critical risks
Contingency and crisis management plans in place and rehearsed regularly
A real option is an option involving real, as opposed to financial, assets. Real assets include land, plant, and machinery.
Real option analysis uses option pricing theory to value capital investment opportunities. An example of a real option
would be the overbuilding of a facility to provide strategic flexibility in the event that demand were to increase faster
than production capacity.
4
www.co s o.o rg
Thought Leadership in ERM | Risk Assessment in Practice | 7
Speed of Onset (or Velocity)

Speed of onset refers to the time it takes for a risk event
to manifest itself, or in other words, the time that elapses
between the occurrence of an event and the point at which
the company first feels its effects. Knowing the speed of

onset is useful when developing risk response plans.
Illustrative Speed of Onset Scale

Rating
Descriptor
Definition
Very High
Very rapid onset, little or no warning, instantaneous
High
Medium
Low
Onset occurs in a matter of several months
Very Low
Very slow onset, occurs over a year or more
Onset occurs in a matter of days to a few weeks

Onset occurs in a matter of a few months
Inherent and Residual Risk

When assessing risks, its important to determine whether
respondents will be asked to assess inherent risk, residual
risk, or both. In Enterprise Risk Management Integrated
Framework (2004), COSO defines inherent risk as the risk to
an entity in the absence of any actions management might
take to alter either the risks likelihood or impact. Residual
risk is the risk remaining after managements response to
the risk. Applying this concept is trickier than it might seem
at first glance. Some entities interpret inherent risk to be
level of risk assuming responses currently in place fail,
and residual risk to be the level of risk assuming existing
responses operate according to design. Other entities

interpret inherent risk to be the current level of risk
assuming existing responses operate according to design
and residual to be the estimated risk after responses
under consideration are put into place. The first approach
is focused more on controls effectiveness of the current
environment and the second approach on evaluating risk
response options. There is no one right answer and either
approach may be useful depending upon the purpose of the
assessment and the nature of the risks being considered.
www.co s o.o rg
Assess Risks
Risk assessment is often performed as a two-stage
process. An initial screening of the risks and opportunities
is performed using qualitative techniques followed by a
more quantitative treatment of the most important risks and
opportunities lending themselves to quantification (not all
risks are meaningfully quantifiable). Qualitative assessment
consists of assessing each risk and opportunity according
to descriptive scales as described in the previous section.
Quantitative analysis requires numerical values for both
impact and likelihood using data from a variety of sources.
The quality of the analysis depends on the accuracy and

completeness of the numerical values and the validity of the
models used. Model assumptions and uncertainty should be
clearly communicated and evaluated using techniques such
as sensitivity analysis.
Both qualitative and quantitative techniques have advantages
and disadvantages. Most enterprises begin with qualitative
assessments and develop quantitative capabilities over time
as their decision-making needs dictate.
Measurement Techniques Comparison

Technique
Advantages
Disadvantages
Qualitative
Is relatively quick and easy

Provides rich information beyond
financial impact and likelihood such as
vulnerability, speed of onset, and
non-financial impacts such as health
and safety and reputation

Is easily understood by a large number
of employees who may not be trained
in sophisticated quantification
techniques
Gives limited differentiation between levels of

risk (i.e. very high, high, medium, and low)
Is imprecise risk events that plot within the
same risk level can represent substantially
different amounts of risk
Cannot numerically aggregate or address risk
interactions and correlations
Provides limited ability to perform cost-benefit
analysis
Quantitative Allows numerical aggregation taking

into account risk interactions when
using an at risk measure such as
Cash Flow at Risk

Permits cost-benefit analysis of risk
response options

Enables risk-based capital allocation
to business activities with optimal
risk-return

Helps compute capital requirements
to maintain solvency under extreme
conditions
Can be time-consuming and costly, especially

at first during model development
Must choose units of measure such as dollars
and annual frequency which may result
in qualitative impacts being overlooked
Use of numbers may imply greater precision
than the uncertainty of inputs warrants
Assumptions may not be apparent
www.co s o.o rg
For qualitative assessments, the most commonly used

assessment techniques are interviews, cross-functional
workshops, surveys, benchmarking, and scenario analysis.
Quantitative techniques range from benchmarking and
scenario analysis to generating forward looking point
estimates (deterministic models) and then to generating
forward looking distributions (probabilistic models).
Some of the most powerful probabilistic models from an
enterprise-wide standpoint include causal at-risk models
used to estimate gross profit margins, cash flows, or
earnings over a given time horizon at given confidence
levels.
Analysis of Existing Data
Reviewing internal and external data can help individuals
assess the likelihood and impact of a risk or opportunity.
Sources of risk occurrence data include internal and
external audit reports, public filings, insurance claims and
internal loss event data including near misses, published
reports by insurance companies, industry consortia, and
research organizations. While relying on existing data
provides objectivity, its important to evaluate the relevance
of the data under current and projected conditions.
Adjustments may be warranted using expert judgment. In
these cases, the rationale for adjustments must be clearly
documented and communicated.
Interviews and Cross-Functional Workshops
Assessment can be conducted through one-on-one
interviews or facilitated meetings. Cross-functional
workshops are preferable to interviews or surveys for
assessment purposes as they facilitate consideration of risk
interactions and break down siloed thinking. Workshops
improve understanding of a risk by bringing together diverse
perspectives. For example, when considering a risk such
as information security breach, workshop participants
from information technology, legal and compliance,
public relations, customer service, strategic planning,
and operations management may each bring different
information regarding causes, consequences, likelihoods,
and risk interactions. Interviews may be more appropriate
for senior management, board members, and senior line
managers due to their time constraints. Workshops may
not work well in cultures that suppress free sharing of
information or divergent opinions.
Surveys
Surveys are useful for large, complex, and geographically
distributed enterprises or where the culture suppresses
open communication. Survey results can be downloaded
into analytical tools allowing risks and opportunities to be
viewed by level (board members, executives, managers),
by business unit, by geography, or by risk category.
Surveys have drawbacks too. Response rates can be low.
If the survey is anonymous, it may be difficult to identify
information gaps. Quality of responses may be low if
respondents give survey questions superficial attention in
a rush to completion, or if they misunderstand something
and dont have the opportunity to ask clarifying questions.
But perhaps most of all, respondents dont benefit from
cross-functional discussions which enhance peoples
risk awareness and understanding, provide context and
information to support the risk ratings, and analyze risk
interactions across silos. For these reasons, surveys
should not be considered a substitute for workshops and
other techniques for in-depth analysis of key risks.
Benchmarking
Benchmarking is a collaborative process among a
group of entities. Benchmarking focuses on specific
events or processes, compares measures and results
using common metrics, and identifies improvement
opportunities. Data on events, processes, and measures
are developed to compare performance. Some companies
use benchmarking to assess the likelihood and impact
of potential events across an industry. Benchmarking
data are available from research organizations, industry
consortia, insurance companies and rating agencies,
government agencies, and regulatory and supervisory
bodies. For example, an oil field services company might
benchmark its safety risk using measures such as lost time
injuries using data for similar companies available from the
Bureau of Labor Statistics, the Occupational Health and
Safety Administration (OSHA), the American Petroleum
Institute (API), or others.
www.co s o.o rg
Scenario Analysis
Scenario analysis has long been recognized for its
usefulness in strategic planning. It is also useful for
assessing risks and tying them back to strategic objectives.
It entails defining one or more risk scenarios, detailing the
key assumptions (conditions or drivers) that determine
the severity of impact, and estimating the impact on a key
objective. In the example below, management wanted to
understand how earnings could be negatively impacted.
Six scenarios impacting earnings were identified, causal

factors (such as price or volume changes or state of the
economy) determined, detailed assumptions calibrated,
and the earnings impact estimated. Scenarios can be
developed jointly by risk owners and ERM personnel
and built out and validated with specialists from various
functions and management.
Scenario Analysis
Scenario Description
Detailed Assumptions
1) Currency changes impact

competitive landscape

2) Natural gas prices increase

15% volume decrease

20% price decrease
Sustained for 9 months
Recovery takes additional 9 months
- $500
$5/MM Btu increase

No ability to pass through increase
- $150
3) Crude oil prices increase

100% increase
Pass through 25% of cost increase
- $15
4) Technology shift

15% volume decrease/year

15% price decrease/year
$2MM less in R&D expenditures
- $275
5) Competitive pressure

10% price decrease

- $200
6) Supply chain disruption

10% volume decrease

- $175
* Earnings before interest and taxes.

Source: Frederick Funston and Stephen Wagner, Surviving and Thriving in Uncertainty
(Hoboken, NJ: John Wiley & Sons, Inc., 2010), 69.
www.co s o.o rg
EBIT* Impact ($MM)
Causal At-Risk Models

Gross Margin at Risk (GMaR), Cash Flow at Risk (CFaR),
and Earnings at Risk (EaR) are metrics built on causal
models where specific risk factors drive future uncertainty
of key cash flow or earnings components. Each risk factor
can be modeled in detail and incorporated into the overall
model. Using a causal at-risk model can provide insight
into how historical relationships might become uncoupled
and deviate meaningfully from expectations. Armed with
the knowledge of how each risk factor could vary in the
future and impact cash flow or earnings, risk can be better
measured and managed. It is the added insight of the risk
factors driving uncertainty that makes causal models a
step up from simply extrapolating past relationships in a pro
forma approach.
Model inputs may be derived from past records, relevant

experience, relevant published literature, market research,
public consultation, experiments and prototypes, and
economic, engineering or other models. Where historical
data are not available, not relevant, or incomplete, expert
elicitation may be used. Expert elicitation is most commonly
used to estimate reasonable probabilities especially for low
likelihood, high impact events. Experts are valuable sources
of information and knowledge. But experts also bring
biases. Fortunately, a large body of knowledge exists with
regard to heuristics and biases and ways to address them.
For example, see COSOs recently issued thought paper,
Enhancing Board Oversight: Avoiding Judgment Traps and
Biases (March 2012).
In reality, both pro forma models built around historical ratios

and causal at-risk models can be helpful and should be seen
as complementary views of an uncertain future. Regardless
of the type of model, the confidence placed on estimates of
levels of risk and assumptions made in the analysis should
be clearly stated.
www.co s o.o rg
Assess Risk Interactions

ERM enables an integrated and holistic view of risks. The
key here is that the whole does not equal the sum of the
parts. To understand portfolio risk, one must understand the
risks of the individual elements plus their interactions due
to the presence of natural hedges and mutually amplifying
risks. Understanding risk interactions and then managing
them requires breaking down silos.
A simple way to consider risk interactions is to group related
risks into a broad risk area (such as grouping risks related to
sourcing, distribution channels, vendor concentrations, etc.
into supply chain risk) and then assigning ownership and

oversight for the risk area. Three explicit ways to capture
risk interactions increasing in level of complexity and
richness of information are risk interaction maps, correlation
matrices, and bow-tie diagrams.
Risk Interaction Map
A risk interaction map is the simplest form of graphical
representation in which the same list of risks form the x and
y axes. Risk interactions are then indicated by an X or other
qualitative indicator.
Supply Chain
Disruption
Customer
Preference Shift
Copper Price
Increase >25%
Work Stoppage
>1 Week
Economic
Downturn
Supplier
Consolidation
Local Competitor
Enters Market
New Substitutes
Available
Cost of Capital
Increase >5%
Tighter Emission
Standards
FCPA
Violation
Exchange Rate
Fluctuations
www.co s o.o rg
Exchange Rate
Fluctuations
FCPA
Violation
Tighter Emission
Standards
Cost of Capital
Increase >5%
New Substitutes
Available
Local Competitor
Enters Market
Supplier
Consolidation
Economic
Downturn
Work Stoppage
>1 Week
Copper Price
Increase >25%
Customer
Preference Shift
Risk
Supply Chain
Disruption
Exhibit 3: Illustrative Risk Interaction Map
Where historical data are available, risk interactions can

be expressed quantitatively using a correlation matrix.
This is an especially useful technique to apply within a risk
category such as market risk. Difficulties in determining
correlations for risks include the possibility that past causal
relationships will not be indicative of future relationships,
lack of historical data, differences in time frames (short-,
medium-, and long-term), and the large numbers of risks
required for an enterprise-wide assessment.
Developing the Full PictureFault Trees,
Event Trees, and Bow-Tie Diagrams
Diagrams that break a complex risk occurrence into its
component parts showing the chains of events that could
lead to or result from the occurrence can be indispensable
for identification and assessment of risk responses and key

risk indicators. The diagrams can be qualitative or serve
as the basis for quantitative models. Three commonly used
diagrams are fault trees, event trees, and bow-ties. Fault
trees are used for analyzing events or combinations of
events that might lead to a hazard or an event. Event trees
are used for modeling sequences of events arising from
a single risk occurrence. A bow-tie diagram combines a
fault tree and an event tree and takes its name from its
shape. Probabilistic models built on bow-tie diagrams are
versatile for quantifying inherent and residual risk levels
and performing what-if, scenario, and sensitivity analyses.
Exhibit 4: Bow-Tie Diagram

Risk Factors
Risk
Consequences
Trigger
Event
Intermediate
Event
End
Event
Consequence
End Event
(Loss)
Trigger
Event
Intermediate
Event
End
Event
Consequence
End Event
(Loss)
Intermediate
Event
End
Event
Consequence
End Event
(Loss)
Intermediate
Event
End
Event
Consequence
End Event
(Loss)
Condition
Intermediate
Event
End
Event
Consequence
End Event
(Loss)
Condition
Intermediate
Event
End
Event
Consequence
End Event
(Loss)
Condition
Intermediate
Event
End
Event
Consequence
End Event
(Loss)
Risk
Note: The terms fault tree, event tree, and bow-tie diagram are sometimes used interchangeably.
www.co s o.o rg
Prioritize Risks
Once the risks have been assessed and their interactions
documented, its time to view the risks as a comprehensive
portfolio to enable the next step prioritizing for risk
response and reporting to different stakeholders. The term
risk profile represents the entire portfolio of risks facing
the enterprise. Some entities represent this portfolio as
a hierarchy, some as a collection of risks plotted on a
heat map. Entities with more mature ERM programs and
quantitative capabilities may aggregate individual risk
distributions into a cumulative loss probability distribution
and refer to that as the risk profile.
Similar to assessing risks, ranking and prioritizing is often
done in a two-step process. First, the risks are ranked
according to one, two, or more criteria such as impact
rating multiplied by likelihood rating or impact multiplied
by vulnerability. Second, the ranked risk order is reviewed

in light of additional considerations such as impact alone,
speed of onset, or the size of the gap between current and
desired risk level (risk tolerance threshold). If the initial
ranking is done by multiplying financial loss by likelihood,
then the final prioritization should take qualitative factors
into consideration.
Hierarchies and Rolling Up and Drilling Down
The simplest way to aggregate risks is to organize them
according to a hierarchy. This is often done in risk
management systems where risks can be organized by
organizational unit, risk type, geography, or strategic
objective. The better systems allow users to roll up and drill
down for analysis and reporting. This provides a complete
listing of the assessed risks but does not help with prioritizing.
Exhibit 5: Risk Hierarchies

Risk Hierarchy by Org. Unit
Enterprise
Risk Hierarchy by Risk Type

Enterprise
Business Unit 1
Strategic
Risk ABC
Risk ABC
Risk DEF
Project 1
Risk UVW
Risk ABC in Bus. Unit 1

Risk ABC in Bus. Unit 2
Financial
Risk DEF
Risk XYZ
Risk DEF in Bus. Unit 1
Project 2
Risk GHI
Risk UVW
Risk XYZ
Business Unit 2
Operational
Risk UVW
Risk UVW in Project 1
Risk ABC
Risk UVW in Project 2
Risk GHI
Risk JKL
Risk GHI in Bus. Unit 2
Risk DEF
Risk DEF in Bus. Unit 1
Compliance
Risk n . . .
www.co s o.o rg
Risk Maps
Another simple way to view the portfolio is to create a
risk map, often called a heat map. These are usually twodimensional representations of impact plotted against
likelihood. They can also depict other relationships such as
impact versus vulnerability. For even richer information, the
size of the data points can reflect a third variable such as
speed of onset or the degree of uncertainty in the estimates.
The most common way to prioritize risks is by designating
a risk level for each area of the graph such as very high,
high, medium, or low, where the higher the combined
impact and likelihood ratings, the higher the overall risk
level. The boundaries between levels vary from entity to
entity depending on risk appetite. For example, an entity
with a greater risk appetite will have boundaries between
risk levels shifted toward the upper right, and an entity with
greater risk aversion will have boundaries between risk
levels shifted toward the bottom left. Also, some entities
adopt asymmetric boundaries placing a somewhat greater
emphasis on impact than on likelihood. For example, a risk
having an impact rating of moderate and likelihood rating
of frequent has an assigned risk level of high, whereas a
risk having an impact rating of extreme and a likelihood
rating of possible has an assigned risk level of very high.
15
After plotting on the heat map, risks are then ranked from
highest to lowest in terms of risk level. These rankings
may then be adjusted based on other considerations such
as vulnerability, speed of onset, or detailed knowledge of
the nature of the impact. For example, within a group of
risks having a designation of very high, those risks having
extreme health and safety or reputational impacts may be
prioritized over risks having extreme financial impacts but
lesser health and safety or reputational impacts.
When using numerical ratings in a qualitative environment,
its important to remember that the numbers are labels and
not suitable for mathematical manipulation although some
entities do multiply the ratings, such as for impact and
likelihood, to develop a preliminary ranking.
Where entities have defined impact scales for both
opportunities and risks, they may plot risks on a map
such as that illustrated in exhibit 6. This allows a direct
comparison of the highest rated opportunities and risks for
consideration and prioritization.
Exhibit 6: Illustrative Combined Risk and Opportunity Map

Impact
Opportunities
Likelihood
Extreme
Major
Risks
Moderate Minor
Incidental Incidental Minor
Moderate
Major
Extreme
Frequent
Likely
Possible
Unlikely
Rare
www.co s o.o rg
Consider the following example: A company identified

60 risks to include in its risk universe. It then determined
appropriate assessors. It used a combination of interviews,
workshops, and a survey to perform an initial qualitative
assessment of impact, likelihood, vulnerability, and speed
of onset criteria. Risk interactions were evaluated for the
highest risks and the assessments were refined. Risks

were plotted on a heat map to perform an initial prioritization.
Twelve risks plotted in the Very High risk level designated as
red in the below heat map. These risks were designated key
risks meaning that they will be reported to and monitored by
executive leadership and the board of directors.
Exhibit 7: Illustrative Heat Map

5
3
10
Likelihood
11
5
n
n n
n
1
1
60
n n
12
n
4
Impact
Dots represent risk #1 - #n
Dot size reflects speed of onset:
Very Low
Low
Medium
www.co s o.o rg
ID
Risk
1
2
3
4
5
6
7
8
9
10
1 1
12
n
60
Supply chain disruption

Customer preference shift
Copper price rise >10%
Work stoppage > 1 week
Economic downturn
Supplier consolidation
Local competitors enter
New substitutes available
Cost of capital rise >5%
Tighter emission standards
FCPA violation
Exchange rate fluctuations
. . .
Impairment of assets
I = Impact
High
Very High
L = Likelihood
4.8
4.1
4.3
4.4
4.0
3.8
3.9
4.5
2.9
3.4
4.0
2.7
...
1.6
3.7
3.3
4.7
4.5
3.7
4.2
4.5
3.6
4.0
4.6
4.0
4.1
...
2.7
3.8
3.5
2.3
4.1
3.5
3.2
3.6
4.2
2.9
2.9
3.3
2.7
...
1.6
4
2
4
3
2
1
1
1
3
1
5
4
...
1
V = Vulnerability S = Speed of onset
Another useful plot for prioritizing is the MARCI chart

(for Mitigate, Assure, Redeploy, and Cumulative Impact),
depicted in exhibit 8. The MARCI chart plots risks along
the two axes of impact and vulnerability, and indicates
each risks speed of onset by the size of the data points.
This is particularly useful when the primary purpose of the
prioritization exercise is for risk response: risks plotting the
farthest in the upper right quadrant represent the highest
impact and vulnerability and would benefit the most from
additional management effectiveness in managing the risks.
Continuing our example, the 12 risks rated Very High were
plotted on a MARCI chart to further refine the prioritization
17
and to perform a preliminary evaluation of the type of

appropriate risk response. In this view, the company can
see how its hedging program reduces its vulnerability to
copper price increases (risk 3), and evaluate its previous
decision to not hedge against currency fluctuations (risk
12). Leadership can also see that supply chain disruption
(risk 1) can occur with little warning and severe impact.
This and the other risks in its quadrant require action
to reduce vulnerability. The executive leadership team
and board members will pay particular attention to
managements actions to respond to these risks. The
top 12 risks were tagged for further quantification and
probabilistic modeling.
Exhibit 8: Illustrative MARCI Chart

5 Assurance of Preparedness
Enhance Risk Mitigation
48
Potential Impact
11
2
5
10
Redeploy Resources
Measure of Cumulative Impact
12
2
ID
Risk
1
2
3
4
5
6
7
8
9
10
1 1
12
Supply chain disruption

Customer preference shift
Copper price rise >10%
Work stoppage > 1 week
Economic downturn
Supplier consolidation
Local competitors enter
New substitutes available
Cost of capital rise >5%
Tighter emission standards
FCPA violation
Exchange rate fluctuations
4.8
4.1
4.3
4.4
4.0
3.8
3.9
4.5
2.9
3.4
4.0
2.7
3.7
3.3
4.7
4.5
3.7
4.2
4.5
3.6
4.0
4.6
4.0
4.1
3.8
3.5
2.3
4.1
3.5
3.2
3.6
4.2
2.9
2.9
3.3
2.7
4
2
4
3
2
1
1
1
3
1
5
4
1
1
Potential Vulnerability
Dots represent risk #1 - #n
Dot size reflects speed of onset:
Very Low
Low
Medium
High
I = Impact
L = Likelihood
V = Vulnerability S = Speed of onset
Very High
Aggregating in a Quantitative Environment

In situations where key risks have been quantified using
a common measure such as financial loss or an atrisk measure, it is possible to aggregate the individual
probability distributions into a single distribution reflecting
correlations and portfolio effects. Measures that are
gaining traction for this purpose are gross margin at risk,
cash flow at risk, and earnings at risk.
The primary applications for a single at-risk measure

presenting an aggregate view of risk (over a given time
horizon at a specified confidence level) are capital allocation,
solvency assessments, and measures of risk utilization and
capacity relative to risk appetite. Risk aggregation models
are extremely variable from one enterprise to another, even
within the financial services industry.
www.co s o.o rg
Putting It into Practice

To be effective and sustainable, the risk assessment
process needs to be simple, practical, and easy to
understand. Success depends upon executive commitment
and resources. The process must be performed by people
with the right skills supported by technology that is correctly
sized for the task at hand.
Fortunately, a large number of software vendors have

entered the ERM space, and each year brings new
innovations and improved offerings. Systems exist at an
array of price points with analytical capabilities increasing
with price. Most systems will quickly pay for themselves in
saved labor costs.
A corporate-level ERM function is indispensable for defining

common standards, coordinating assessments across
business units, and facilitating analysis of risk interactions.
The central ERM function must be staffed by people with
the necessary facilitation, project management, and
analytical skills along with knowledge of risk management
leading practices. The ERM function must be augmented by
people in line positions closest to the risks. The risk owners
ultimately bear responsibility for the assessed levels of risk
and defining and implementing risk response plans to bring
risks within tolerance. This hybrid top-down and bottomup approach brings the best of both worlds achieving
consistency and comprehensive coverage while embedding
accountability and leveraging expertise of the people in the
organization closest to the risks.
Finally, risk assessment cannot exist in a vacuum or it

becomes a fruitless exercise. COSOs Enterprise Risk
Management Integrated Framework emphasizes
the need to assess and oversee risks from a holistic
perspective. The process must sit within a larger
framework that uses the information gleaned to make
decisions about risk responses and monitoring, and feeds
information back into the strategic planning process.
The ERM function must be empowered to monitor and
oversee implementation of risk responses. If participants
dont see that their contributions and hard work during
risk assessment lead to concrete actions that make a real
difference, they will become cynical and withdraw from the
process in future years.
People arent enough. To be efficient, they must be

supported by the right technology. Many entities begin
their ERM journey in a simple spreadsheet environment.
This can be practical in the early stages of development
as both risk owners and senior leadership ascertain their
analytical and reporting requirements. Later years can
be quite challenging without automation, especially if the
entity is large, complex, and geographically distributed.
www.co s o.o rg
Youll know youre doing risk assessment right when

leaders at every level use the information to make
decisions regarding value.
19
About COSO
Originally formed in 1985, COSO is a joint initiative of five private sector organizations and is dedicated to providing thought
leadership through the development of frameworks and guidance on enterprise risk management (ERM), internal control,
and fraud deterrence. COSOs supporting organizations are the Institute of Internal Auditors (IIA), the American Accounting
Association (AAA), the American Institute of Certified Public Accountants (AICPA), Financial Executives International (FEI),
and the Institute of Management Accountants (IMA).
About the Authors

Deloitte is the brand under which tens of thousands of dedicated professionals in independent firms throughout the
world collaborate to provide audit, consulting, financial advisory, risk management and tax services to selected clients.
These firms are members of Deloitte Touche Tohmatsu Limited (DTTL), a UK private company limited by guarantee. In
the United States, Deloitte LLP is the member firm of DTTL. Deloitte & Touche LLP, a subsidiary of Deloitte LLP, provides
internal control and enterprise risk services in the United States. Please see www.deloitte.com/us/about for a detailed
description of the legal structure of Deloitte LLP and its subsidiaries. Certain services may not be available to attest
clients under the rules and regulations of public accounting.
The contributing authors from Deloitte & Touche LLP are Dr. Patchin Curtis, Director, and Mark Carey, Partner.
This publication contains general information only and Deloitte is not, by means of this publication, rendering accounting,
business, financial, investment, legal, tax, or other professional advice or services. This publication is not a substitute
for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your
business. Before making any decision or taking any action that may affect your business, you should consult a qualified
professional advisor.
Deloitte shall not be responsible for any loss sustained by any person who relies on this publication.
www.co s o.o rg
C o m m i t t e e
o f
S p o n s o r i n g
o f
t h e
T r e a d w a y
C o m m i s s i o n
D evelo p i n g
K e y
R isk
I n dic a tors
to
S tre n g the n
E n ter p rise
R isk
M a n a g eme n t
How Key Risk Indicators can Sharpen Focus

on Emerging Risks
By
Mark S. Beasley | Bruce C. Branson | Bonnie V. Hancock
Authors
ERM Initiative at North Carolina State University
Mark S. Beasley
Deloitte Professor of Enterprise Risk Management

Bruce C. Branson
Associate Director, ERM Initiative
The ERM Initiative at North Carolina State University is pioneering

thought-leadership about the emergent discipline of enterprise
risk management, with a particular focus on the integration of ERM
in strategy planning and governance. The ERM Initiative conducts
outreach to business professionals through executive education
and its internet portal (www.erm.ncsu.edu); research, advancing
knowledge and understanding of ERM issues; and undergraduate
and graduate business education for the next generation
of business executives.
Bonnie V. Hancock
Executive Director, ERM Initiative
COSO Board Members

David L. Landsittel
COSO Chair
Larry E. Rittenberg
COSO Chair - Emeritus
Mark S. Beasley
Chuck Landes
American Institute of Certified Public Accountants
Richard F. Chambers
Jeff Thomson
Marie Hollein
Preface
This project was commissioned by COSO, which is dedicated to providing thought leadership
through the development of comprehensive frameworks and guidance on enterprise risk
management, internal control, and fraud deterrence designed to improve organizational
performance and governance and to reduce the extent of fraud in organizations. COSO is a
private sector initiative, jointly sponsored and funded by the following organizations:

American Institute of Certified Public Accountants (AICPA)
Institute of Management Accountants (IMA)

www.co s o.o rg
D evelo p i n g
K e y
R isk
I n dic a tors
to
S tre n g the n
E n ter p rise
R isk
M a n a g eme n t
How Key Risk Indicators can Sharpen Focus

on Emerging Risks
December 2010
1 2 3 4 5 6 7 8 9 0 PIP 19876543210
American Institute of Certified Public Accountants, licensing and permissions agent for COSO copyrighted materials.
Direct all inquiries to copyright@aicpa.org or to AICPA, Attn: Manager, Rights and Permissions, 220 Leigh Farm Rd.,
www.co s o.o rg
Thought Leadership in ERM | Developing Key Risk Indicators to Strengthen Enterprise Risk Management | III
Introduction
Boards of directors have become increasingly aware
of their responsibilities related to effective oversight
of managements execution of enterprise-wide risk
management processes. This is due, in part, to significant
external pressures that have developed recently that
are thrusting risk management and its oversight to the
forefront of many board agendas and management action
plans. For example, the New York Stock Exchange in 2004
adopted governance rules that require audit committees of
NYSE-listed firms to oversee managements risk oversight
processes. In 2008, Standard & Poors began explicitly
evaluating an issuers enterprise risk management (ERM)
processes in seventeen new industries, as an additional
component of their credit ratings analysis. In 2009, the
Securities and Exchange Commission (SEC) expanded
proxy disclosure requirements to increase information for
investors about the boards role in risk oversight. The 2010
Federal Financial Reform legislation now mandates risk
committees for boards of financial institutions and other
entities overseen by the Federal Reserve.
Many organizations are embracing an enterprise-wide
approach to risk oversight known as enterprise risk
management (ERM) and executive management teams
leading these efforts are turning to frameworks, such as
COSOs 2004 Enterprise Risk Management Integrated
Framework (COSO ERM Framework), to aid them in
strengthening their enterprise-wide risk management
processes.
COSOs ERM Framework defines ERM as follows:

As indicated by this definition, ERM provides the opportunity

for organizational leaders to achieve a robust and holistic
enterprise-wide view of potential events that may affect the
achievement of the organizations objectives. Because risks
are constantly evolving as an organization strives to achieve
its objectives, there is a high demand for relevant and timely
risk information.
Many organizations are seeking to develop a process that
provides management and the board of directors with
rich information about potential events that may affect the
entity, especially top risk exposures, that they can monitor
on an ongoing basis. While most organizations monitor
numerous key performance indicators (KPIs), often those
indicators shed insights about risk events that have already
affected the organization. Increasingly, boards and senior
executives are looking to develop metrics or indicators to
help to better monitor potential future shifts in risk conditions
or new emerging risks so that management and boards
are able to more proactively identify potential impacts
on the organizations portfolio of risks. Doing so enables
management and the board to be in a better position to
manage events that may arise in the future on a more timely
and strategic basis. This latter type of metric or indicator is
frequently referred to as a key risk indicator (KRI).
The purpose of this thought paper is to help management
develop effective key risk indicators (KRIs) to heighten board
and management enterprise risk awareness in order to
increase the effectiveness of an ERM process and improve
the execution of an organizations strategy.
Enterprise risk management is a process, effected by

an entitys board of directors, management, and other
personnel, applied in strategy setting and across the
enterprise, designed to identify potential events that
may affect the entity, and manage risk to be within the risk
appetite, to provide reasonable assurance regarding the
achievement of entity objectives.
www.co s o.o rg
iv
| Developing Key Risk Indicators to Strengthen Enterprise Risk Management | Thought Leadership in ERM
Content Outline
Page
Description
Differentiating Key Performance Indicators
from Key Risk Indicators
Developing Effective Key Risk Indicators
KRIs Provide Opportunities for Proactive

Strategic Risk Management
Sources of Information When Developing KRIs
KRI Communication and Reporting:

Role of the Board, Management, and Risk Owners
The Value Proposition for Key Risk Indicators
10
Summary Observations
11
About COSO
12
About the Authors
12
www.co s o.o rg
Thought Leadership in ERM | Developing Key Risk Indicators to Strengthen Enterprise Risk Management |
Differentiating Key Performance Indicators from Key Risk Indicators

It is important to distinguish key performance indicators
(KPIs) from key risk indicators (KRIs). Both management
and boards regularly review summary data that include
selected KPIs designed to provide a high-level overview of
the performance of the organization and its major operating
units. These reports often are focused almost exclusively
on the historical performance of the organization and
its key units and operations. For example, reports often
highlight monthly, quarterly, and year-to-date sales trends,
customer shipments, delinquencies, and other performance
data points relevant to the organization. It is important to
recognize that these measures may not provide an adequate
early warning indicator of a developing risk because they
mostly focus on results that have already occurred.
While KPIs are important to the successful management of
an organization by identifying underperforming aspects of
the enterprise as well as those aspects of the business that
merit increased resources and energy, senior management
and boards also benefit from a set of KRIs that provide
timely leading-indicator information about emerging risks.
Measures of events or trigger points that might signal
issues developing internally within the operations of the
organization or potential risks emerging from external
events, such as macroeconomic shifts that affect the
demand for the organizations products or services, may
provide rich information for management and boards to
consider as they execute the strategies of the organization.
Key risk indicators are metrics used by organizations to

provide an early signal of increasing risk exposures in
various areas of the enterprise. In some instances, they
may represent key ratios that management throughout
the organization track as indicators of evolving risks, and
potential opportunities, which signal the need for actions
that need to be taken. Others may be more elaborate and
involve the aggregation of several individual risk indicators
into a multi-dimensional score about emerging events that
may lead to new risks or opportunities.
An example related to the oversight of accounts receivable
collection helps illustrate the difference in KPIs and KRIs.
A key performance indicator for customer credit is likely to
include data about customer delinquencies and write-offs.
This key performance indicator, while important, provides
insights about a risk event that has already occurred (e.g.,
a customer failed to pay in accordance with the sales
agreement or contract). A KRI could be developed to help
anticipate potential future customer collection issues so that
the credit function could be more proactive in addressing
customer payment trends before risk events occur. A
relevant KRI for this example might be analysis of reported
financial results of the companys 25 largest customers or
general collection challenges throughout the industry to see
what trends might be emerging among customers that could
potentially signal challenges related to collection efforts in
future periods.
Objective
Manage the collection of accounts receivable to reduce loss due to write-offs
Key Performance Indicator (KPI)
Key Risk Indicator (KRI)
Data about write-offs of accounts in most recent

month, quarter, year.
Analysis of reported financial results for the

companys 25 largest customers or general collection
challenges throughout the industry that highlight
trends signaling future collection concerns.
www.co s o.o rg
Developing Effective Key Risk Indicators

A goal of developing an effective set of KRIs is to identify
relevant metrics that provide useful insights about potential
risks that may have an impact on the achievement of
the organizations objectives. Therefore, the selection
and design of effective KRIs starts with a firm grasp of
organizational objectives and risk-related events that might
affect the achievement of those objectives. Linkage of top
risks to core strategies helps pinpoint the most relevant
information that might serve as an effective leading indicator
of an emerging risk.
In the simple illustration below, management has an
objective to achieve greater profitability by increasing
revenues and decreasing costs. They have identified four

strategic initiatives that are critical to accomplishing those
objectives. Several potential risks have been identified that
may have an impact on one or more of four key strategic
initiatives. Mapping key risks to core strategic initiatives
puts management in a position to begin identifying the most
critical metrics that can serve as leading key risk indicators
to help them oversee the execution of core strategic
initiatives. As shown below, KRIs have been identified for
each critical risk. Mapping KRIs to critical risks and core
strategies reduces the likelihood that management becomes
distracted by other information that may be less relevant to
the achievement of enterprise objectives.
Linking Objectives
Linking Objectives to Strategies to Risks To KRIs
Strategic
Initiative #1
Increase
Revenues
Strategic
Initiative #2
Profitability
Strategic
Initiative #3
Reduce
Costs
Strategic
Initiative #4
To illustrate further, consider a simple example involving a

chain of family-style buffet restaurants. Management is
interested in avoiding a negative earnings event that could
arise due to unexpected market conditions that might
negatively affect revenues. They know that restaurant
traffic is directly affected by the availability of customer
discretionary income. As discretionary income levels fall
off, customers are less likely to dine outside their homes.
A key metric that management uses as a leading indicator
of potential changes in customer discretionary income
levels is average gasoline prices people pay at the pump.
Management has determined that when gasoline prices
spike (or are expected to rise), discretionary income for
individuals and families representing their core customer
www.co s o.o rg
Potential
Risk
KRI
Potential
Risk
KRI
Potential
Risk
KRI
Potential
Risk
KRI
Potential
Risk
KRI
base decreases. When gas prices rise rapidly or are

forecasted to stay at unusually high levels, customer traffic
begins to drop.
Management has found that close monitoring of forecasts
of per-gallon prices of gas in the chains geographic
market and trends in oil futures prices help management
proactively identify early indicators of potential changes
in customer visits. Monitoring these key risk metrics
provides management the opportunity to proactively modify
sales strategies by adjusting marketing and restaurant
promotion events thereby reducing the impact of the risk as
discretionary income begins to decline.
Example
A buffet-style restaurant chain monitors gas prices to identify sales and profitability trends
that may signal the need for modifications to sales strategies.
Objective
Strategic Initiative Potential Risks
Key Risk Indicators
Strategic Response
Increase earnings
through revenue
increases.
Promote premium
buffet options to
attract additional
customers.
Trends in pergallon gasoline

prices in the chains
geographic markets
Revise marketing
to promote more
value options
if gasoline price
trends are rising.
Customer
income levels
and discretionary
income drop and
prevent customers
from visiting
restaurants or from
selecting premium
buffet options.
An effective method for developing KRIs begins by analyzing

a risk event that has affected the organization in the past
(or present) and then working backwards to pinpoint
intermediate and root cause events that led to the ultimate
loss or lost opportunity. The goal is to develop key risk
indicators that provide valuable leading indications that risks
Trends in oil futures

prices
may be emerging. The closer the KRI is to the ultimate root

cause of the risk event, the more likely the KRI will provide
management time to proactively take action to respond to
the risk event. This process can be depicted visually in the
following manner.
Leading Indicators of Risk Event

Leading Indicators of Risk Event
Potential
Risk
Risk Event
Intermediate Event
Leading Indicators of Event?
Root Cause Event
Leading Indicators of Event?
In this diagram, the passage of time proceeds from a root

cause event to (potentially) an intermediate event that
ultimately leads to a risk event. In developing a KRI to serve
as a leading indicator for potential future occurrences of
this risk, it can be helpful to think through the chain of events
that led to the loss so that management can uncover the
ultimate driver (i.e., root cause(s)) of the risk event.
Management can then use that analysis to identify

information associated with the root cause event or
intermediate event that might serve as a key risk indicator
related to either event. When KRIs for root cause events and
intermediate events are monitored, management is in an
enviable position to identify early mitigation strategies that
can begin to reduce or eliminate the impact associated with
an emerging risk event.
www.co s o.o rg
As an illustration, lets assume that management is concerned

about the risk that the organization may breach covenants
associated with its outstanding debt. In this example, a
covenant breach would represent the risk event that is of
concern. In developing effective KRIs to help management
monitor the risk of default, they may look backwards to
identify potential intermediate events that may arise before
the organization reaches the point of a covenant breach.
For example, an intermediate event preceding a possible
covenant breach might involve decreases in sales in recent
months (i.e., covenants based on net income or interest
coverage). Additionally, shortages of cash or increases in the
need for short-term borrowings or draws under existing
lines-of-credit may provide early warning signs that a
covenant breach may be looming in the near term. Key risk
indicators that help monitor these intermediate events put
management in a better position to implement potential
mitigation strategies, such as earlier discussions with key
lenders before an actual covenant breach has occurred.
But, only monitoring KRIs tied to intermediate events allows

less time for management to proactively manage the
emerging risk event than would be the case if management
had access to KRIs related to earlier root cause events
that often precede intermediate events. In this example,
external data, such as customer industry reports and
economic indicators, combined with internal data, such as
input pricing trends, labor issues, plant capacity, key staff
turnover, among other KRIs may provide useful leading
indicators of conditions that may likely initiate events, such
as future drops in sales or future cash shortages that will
lead to an intermediate event and ultimately to the actual
risk event of covenant default. In addition, these key risk
indicators may highlight potential opportunities to increase
sales or improve operations that management may wish to
capture.
The following figure illustrates the linkage of KRIs to both
root cause events and intermediate events.
KRIs
to Inform About Risk of Debt Covenant Default
Example
KRIs to Inform About Risk of Debt Covenant Default
Potential
Risk
Risk Event
Debt covenant breach
Intermediate Event
Leading KRIs might include sales trends, cash on hand,
changes in short-term borrowings, etc.
Root Cause Event
Leading KRIs might include customer financial reports, industry reports,
economic conditions, pricing trends, labor issues, plant capacity, etc.
KRIs Provide Opportunities for Proactive Strategic Risk Management

A well-designed ERM system provides information that allows
management to understand whether key strategic objectives
are being met and to identify opportunities to adjust strategies
and tactics to take advantage of shifts in the environment that
might be exploited for the benefit of the organization and its
www.co s o.o rg
stakeholders. As illustrated by the figure on the next page,

management selects initial strategies at a point in time. As
time goes by, the range of uncertainty begins to increase,
threatening the successful execution of those strategies.
Thought Leadership in ERM | Developing Key Risk Indicators to Strengthen Enterprise Risk Management | 5
To help monitor risks that unfold due to that uncertainty,

trigger points are established with action plans pinpointed
management has identified various KRIs that they are
in advance.
monitoring as they execute the chosen strategic initiatives.
In advance, management has pre-determined certain
This strategic use of KRIs increases the likelihood that goals
levels or thresholds for each KRI that will trigger actions by
and objectives set by management are achieved due to the
management to adjust their strategies proactively to manage fact that risks and the related strategies are managed more
the riskKRIs
accordingly.
Once strategies
are revised, new
KRI
proactively when
KRIs have Risks
been identified.
Facilitate
Proactive
Management
of relevant
Emerging
Trigger Points
KRIs
Trigger Points
KRIs Facilitate Proactive Management of Emerging Risks
KRIs
Uncertainty
Increases
with Longer
Time Horizons
Time
Initial Strategies
Revise Strategies
Revise Strategies
Sources of Information When Developing KRIs

Virtually all organizations possess existing risk metrics that
have evolved over time. These metrics should be carefully
evaluated for their efficacy and continue to be employed if
found to be valuable in highlighting potential emerging risks.
Augmenting these existing KRIs with new metrics is likely to
be required, however.
The KRI identification process may benefit from subject
matter experts within the organization as these individuals
may be in the best position to know where stress points
(i.e., root cause events and intermediate events) exist in
the units they manage or processes they oversee. Their
input helps ensure that key risks are not overlooked and
that KRIs designed to highlight these risks or trends are
most likely to be effective in communicating an early
indication of necessary action. One caution to note is that
these individuals may be biased towards existing risk
metrics already in use, and that they are comfortable with,
at the expense of possibly improved measures that require
additional analysis and validation before adoption.
Another important element in designing effective KRIs

involves the assurance that all parties involved in collecting
and aggregating KRI data are clear about definitions of
individual data items to be captured and any conversion
or standardization methodology to be utilized. Without
confidence in the uniformity of the KRI measurement
approach, aggregated information will lack robustness
and introduce noise into the ultimate decision process. For
example, if customer financial conditions are to be captured
across business units as a KRI, it will be important to
carefully define how that is to be measured. In this scenario,
the following questions may need to be addressed. Should
all customers be equally weighted? Should customer size/
volume of business be a factor? How much time must
elapse before a customer is deemed to be in a difficult
financial state? Are any customers shared by more than one
business unit? If so, which unit makes the determination?
www.co s o.o rg
An important element of any KRI is the quality of the

available data used to monitor a specific risk. Attention must
be paid to the source of the information, either internal to
the organization or drawn from an external party. Sources of
information are likely to exist that can help inform the choice
of KRIs to be employed. For example, internal data may be
available related to prior risk events that can be informative
about potential future exposures. However, internal data
is typically unavailable for many risksespecially those
that have not been encountered previously. And, often risks
likely to have a significant impact may arise from external
sources, such as changes in economic conditions, interest
rate shifts, or new regulatory requirements or legislation.
Thus, many organizations discover that relevant KRIs are
often based on external data, given that many root cause
events and intermediate events that affect strategies arise
from outside the organization.
External sources such as trade publications and loss
registries compiled by independent information providers
may be helpful in identifying potential risks not yet
experienced by the organization. Discussions with key
stakeholders such as customers, employees and suppliers
may provide important insights into risks they face that
may ultimately create risks for the organization. A careful
understanding of regulatory and legal requirements that
must be fulfilled is likely to be helpful in anticipating potential
risks and events that precede them.
KRI data sourced from external and/or independent parties
provides the benefit of objectivity. External/independent
parties are not necessarily unaffiliated with the organization,

but are removed from the business unit from which the KRI is
measured. Almost certainly, trade-offs will be required in this
area. Those individuals charged with ongoing management
of a particular risk are the least objective source (but at
times may be the only available resource for the data
required to produce the KRI in question). A careful validation
of external sources is desirable to enhance confidence in
the ultimate effectiveness of the KRI built from that data.
It is unlikely that a single KRI will adequately capture all
facets of a developing risk or risk trend. For this reason, it is
helpful to analyze a collection of KRIs simultaneously to help
form a better understanding of the risk being monitored. That
said, some KRIs are likely to possess superior predictive
power over other risk metrics and it will be important
to weight each piece of information to reflect its past
performance in forecasting a risk event. Some have referred
to this process as assembling a mosaic of information that
collectively can best provide the early warning of potential
threats developing over time. Realistically, substantial
judgment and experience must be brought to bear on this
process to extract the most meaningful inferences. As the
use of KRIs evolves in an organization, opportunities for
making these judgments will likely yield improvements in KRI
performance.
The following graphic summarizes core elements of welldesigned KRIs.
Based on established practices or benchmarks

Developed consistently across the organization

Provide an unambiguous and intuitive view of the highlighted risk
Allow for measurable comparisons across time and business units
Provide opportunities to assess the performance of risk owners on a timely basis
Consume resources efficiently
An effective way to get started is to take the top 5-10 most

significant risks the organization faces, and charge each risk
owner (the person with primary management responsibility
for a given risk) with the task of identifying one or two
KRIs for their assigned risks. Often, there will be initial
www.co s o.o rg
confusion as to the difference between key performance

indicators that are currently being tracked and KRIs. It will
be important to provide an example or two to help the risk
owners make this distinction.
In the following table, several KRIs are illustrated for a

set of hypothetical risks faced by a regional grocery store
chain seeking to grow earnings by adding new stores in
the Washington, DC and surrounding areas. The company
acquires and develops real estate properties where the
grocery store serves as the anchor tenant alongside other
smaller retail outlets. Acquisition and development of store

properties are contingent on the companys ability to obtain
favorable financing. While these are unique to a particular
business context, they nicely portray the goal of developing
anticipatory data to actively monitor important risks facing
this enterprise.
Example
Regional grocery store chain seeks to grow earnings by adding new stores in Northern
Virginia and Washington, DC area.
Risk Events
Sample KRIs to Monitor Risk Proactively
1. Economic downturn in
Washington, DC markets
affect retail storefront
rental demand and real
estate values
Actual and projected retail store occupancy rates in the

Washington, DC market
Commercial real estate rental market information about leasing prices
and options for similar quality retail properties in the
Washington, DC area.
2. Competition increases
in the Washington, DC
markets
Change in number of grocery stores in market area

Announcements of expansions by big-box retailers and superstores
Significant and sustained price reductions by grocery competitors
in the Washington, DC area
3. Cost of financing
too high
Spreads on debt issuances for comparably rated companies

Actual and projected interest rates
Company stock performance and related trends in competitor stock

4. Delays in developing
property and opening
stores
Compare actual construction and store opening benchmark dates to

pre-determined target dates
Monitor construction labor union issues, including competing demands for
construction labor that might arise due to other major construction projects
in Washington, DC area
5. Long term economic

downturn results in
deteriorating customer
base
Employment outlook for federal government agencies and government

supportive businesses
Forecasts related to unemployment
Consumer spending trends in Washington, DC area
KRI Communication and Reporting: Role of the Board, Management, and Risk Owners
As is true for the larger goal of implementing an enterprise
risk management process in general, the development
and implementation of a set of KRIs requires sensitivity
to organizational culture and a strong message of the
importance of this task from top management and the
board of directors. Creating buy-in from those individuals
within the organization that have day-to-day management
responsibility for various risks will be necessary.
The primary beneficiary of KRIs will be the risk owners
themselves. They will have a set of predictive tools that
should allow them to better manage their business units
to meet goals and objectives set for that unit. Senior
management and boards of directors do not need to know,

nor are they necessarily in a position to fully appreciate,
all KRIs employed within the organization, but they should
be expected to understand and be kept updated on KRIs
related to the organizations top risk exposures. The person
charged with oversight of the enterprise risk management
process can work in concert with the risk owners to identify
appropriate trigger points and action or treatment plans to
be initiated in the event those points are reached. Exception
reports can be developed on a regular basis, the timing of
which will likely vary as a function of the level within the
organization at which the recipient(s) reside.
www.co s o.o rg
8 | Developing Key Risk Indicators to Strengthen Enterprise Risk Management | Thought Leadership in ERM
Senior management may need to review KRI data for risks

and opportunities with significant potential impact to the
organization. Likewise, boards of directors may only require
updates of the most significant KRI data in order to be
confident that the risk management process is functioning
as designed and approved. Dashboard reports that visually
display KRI data overlaying established trigger points can
provide both an intuitive and effective approach to providing
this information to boards. As well, the simple use of color to
depict the status of certain KRIs can quickly highlight those
that require management attention. Green, yellow, and red
are common choices to display conditions associated with
being on target to meet plan goals, in some danger of not
meeting plan goals, and not meeting plan goals, respectively.
KRI
Status
Trend
Retail Occ
RE Rental Market
Change in Stores
Big Box Exp
Price Comp
Debt Spreads
Interest Rates
Stock Perf.
Constr. Progress
These diagrams illustrate examples of effective visual

displays of KRI data for the regional grocery store chain
described previously. The table on the right gives a quick
status and trend of each of the KRIs, and then the two
charts below and on the next page provide more detailed
information for the two KRIs that indicate that plan goals
are not being met. This type of high level report would
be appropriate for communicating KRIs to the board of
directors or senior management.
Labor Market
Govt. Emplmt.
Unemployment
Cons. Spending
Unemployment Rate
8
8.0
7.9
7.8
7.9
7.9
7.8
7.7
7.5
7.5
7.5
Actual
Projected
7.0
80
80
81
Retail Occupancy %
90
88
88
86
85
85
85
84
82
80
82
Actual
80
Projected
75
www.co s o.o rg
7.5
7.5
7.5
Actual
Projected
7.0
80
80
81
Retail Occupancy %
90
88
88
86
85
85
85
84
82
80
82
Actual
80
Projected
75
Also see COSOs 2004 Enterprise Risk Management

Integrated Framework, Volume 2, Application Techniques
for additional examples of dashboard reports.
It is also important to consider the frequency of reporting
KRIs. The appropriate time horizon is dependent upon the
primary user of a specific KRI. For operational managers,
real-time reporting is attractive. For senior management,
where a compilation of KRIs that highlights potential
deviations from organization-level targets is the likely goal, a
less frequent (e.g., monthly) status report may be sufficient.
At the board level, the reporting is often aggregated to allow
for a more strategic evaluation of the data. It is important to
remember that a KRI does not manage or treat risk, and can
lead to a false sense of security if poorly designed. Ideally,
active assessment of the predictive-ability of each KRI is
an ongoing facet of the organizations ERM process.
Once an initial set of KRIs has been designed and

deployed, it is vital that monitoring occurs to validate their
effectiveness. Even well-designed and effective KRIs can
lose value as organizational objectives and strategies
adapt to an ever-changing environment. There is a very
real danger, once a network of KRIs has been established,
that management devotes resources elsewhere within the
organization and ignores the need to refine and replace
existing risk metrics to better capture the data relevant to
the new environment. As part of the initial development and
deployment phase, attention should be paid to the process
that will be followed to continuously track KRI performance.
www.co s o.o rg
The Value Proposition for Key Risk Indicators

The development of KRIs can provide relevant and timely
information to both the board and senior management,
which is significant to effective risk oversight. Effective KRIs
are most often found when they are developed by teams
that include the professional risk management staff and
business unit managers with a deep understanding of the
core operations and strategies of the business subject to
potential risks. Ideally, KRIs are developed in concert with
strategic plans for individual business units and incorporate
acceptable deviations from plan that fall within the overall
risk appetite of the organization.
Effective KRIs can provide value to the organization in a
variety of ways. Potential value may be derived from each of
the following contributions:
Risk Appetite KRIs require the determination of
appropriate thresholds for action at different levels within
the organization. In the grocery chain example, the
unemployment KRI would have a predetermined level at
which the organizations appetite for the risk associated
with the expansion strategy would be exceeded. By
mapping KRI measures to identified risk appetite and
tolerance levels, KRIs can be a useful tool for better
articulating the risk appetite that best represents the
organizational mindset.
Risk and Opportunity Identification KRIs can
be designed to alert management to trends that may
adversely affect the achievement of organizational
objectives or may indicate the presence of new
opportunities. In the grocery chain example, if retail
occupancy levels increase significantly, it may indicate
an opportunity for more development.
Risk Treatment KRIs can initiate action to mitigate
developing risks by serving as triggering mechanisms
for organizational units charged with monitoring particular
KRIs. As well, KRIs can serve as controls by defining limits
to certain actions. In the grocery chain example, there
may be a point at which unemployment reaches such a
high level that the risk of moving forward with expansion
exceeds the organizations appetite and therefore that
KRI level would trigger a revision to the strategy of store
expansion.
www.co s o.o rg
Risk Reporting By design, KRIs can provide

measurable data conducive to aggregation. Summary
reports, as shown earlier for the grocery chain example,
can be quickly communicated to appropriate senior
managers and board members with oversight
responsibilities.
Compliance Efforts For organizations subject to
regulatory oversight, KRIs may be useful in demonstrating
compliance with established requirements in areas such
as capital adequacy or reserve levels.
KRIs designed to assist the board and executive
management in anticipating trends in potential risk-related
events can add considerable value to enterprise-wide risk
oversight efforts by positioning the board and management
so that they can proactively adjust strategies in advance of
or in response to risk events.
In making the business case for KRI development, there are
several examples of benefits that may be obtained:
Improved Performance The use of KRIs to
anticipate emerging risks and shifts in risks over time
can reduce losses, identify opportunities for strategic
exploitation, and potentially reduce the cost of capital by
mitigating perceptions of risk borne by capital providers.
Improved Processes KRIs hold promise in helping
reduce service disruptions, supply chain management,
and enhancing customer experiences by potentially
avoiding certain decisions that unexpectedly create risks
associated with these processes.
Improved Workplace Environment The use
of KRIs can lead to fewer episodes of crisis management,
where normal tasks must be set aside for full-time
devotion to a developing issue. This allows for a more
stable and smoothly functioning organization.
Said differently, a robust set of KRIs should help reduce the
likelihood of surprises and position management and boards
in a proactive versus reactive stance.
Summary Observations
KRIs are metrics used to provide an early signal of
increasing risk exposure in various areas of the organization.
In some instances, they may be little more than key ratios
that the board and senior management track as indicators
of evolving problems, which signal that corrective or
mitigating actions need to be taken. Other times, they may
be more elaborate, involving the aggregation of several
individual risk indicators into a multi-dimensional risk score
about emerging potential risk exposures. KRIs are typically
derived from specific events or root causes, identified
internally or externally, that can prevent achievement of
strategic objectives. Examples can include items such as
the introduction of a new product by a competitor, a strike
at a suppliers plant, proposed changes in the regulatory
environment, or input-price changes.
An executive summary of COSOs Enterprise Risk

ManagementIntegrated Framework provides an
overview of the key principles for effective enterprise risk
management and is available for free download at
www.coso.org. More detailed guidance, including
examples about effective implementation of key ERM
principles, is contained in the full two-volume set.
The design and roll-out of a set of KRIs is an important

element of an organizations enterprise risk management
process. This paper has identified the potential benefits of
developing a set of KRIs, important design elements of those
KRIs, and an appropriate methodology for communicating
KRI data to members of senior management and the
board. Examples of specific KRIs have been provided to
help differentiate them from key performance indicators
that are commonly employed by many organizations. As
organizations look to enhance their risk management
approach, the addition of KRIs to complement existing risk
identification methods will likely yield significant benefits.
www.co s o.o rg
About COSO
The Committee of Sponsoring Organizations of the Treadway Commission (COSO) is a voluntary private-sector organization
comprised of the following organizations dedicated to guiding executive management and governance participants
towards the establishment of more effective, efficient, and ethical business operations on a global basis. It sponsors and
disseminates frameworks and guidance based on in-depth research, analysis, and best practices.
COSO, 2010
About the Authors

Mark S. Beasley, CPA, Ph.D., is the Deloitte Professor of Enterprise Risk Management and director of the ERM
Initiative at North Carolina State University (see www.erm.ncsu.edu). He specializes in the study of enterprise risk
management, corporate governance, financial statement fraud, and the financial reporting process. He is a board member
of the Committee of Sponsoring Organizations of the Treadway Commission (COSO), served on the Conference Boards ERM
Working Group and frequently works with boards and senior executives as they implement ERM. He earned his Ph.D. at
Michigan State University.
Bruce C. Branson, Ph.D., is a professor of accounting and associate director of the Enterprise Risk Management (ERM)
Initiative at North Carolina State University. His teaching and research is focused on financial reporting and includes an
interest in the use of derivative securities and other hedging strategies for risk reduction/risk sharing. He also has examined
the use of various forecasting and simulation tools to form expectations used in financial statement audits and in earnings
forecasting research. He earned his Ph.D. at Florida State University.
Bonnie V. Hancock, M.S., is the executive director of the Enterprise Risk Management (ERM) Initiative, and is also
an executive lecturer in accounting at NC States College of Management. Her background includes executive positions
at both Progress Energy and Exploris Museum. She has served as president of Exploris, and at Progress Energy, has held
the positions of president of Progress Fuels (a Progress Energy subsidiary with more than $1 billion in assets), senior vice
president of finance and information technology, vice president of strategy and vice president of accounting and controller.
She currently serves on the board of directors for AgFirst Farm Credit Bank and Powell Industries.
www.erm.ncsu.edu
www.co s o.o rg
Original Article
A new approach to risk: The

implications of E3
Robert Beaa,*, Ian Mitroffb, Daniel Farberc,
Howard Fosterd and Karlene H. Robertse
a
Department of Civil and Environmental Engineering, University of California,

Berkeley, CA, USA.
E-mail: bea@ce.berkeley.edu
b
Marshall Goldsmith School of Management, Alliant International University,
University of California, Berkeley, CA, USA .
E-mail: ianmitroff@earthlink.net
c
The California Center for Environmental Law and Policy, University of
California, Berkeley, CA, USA .
E-mail: dfarber@law.berkeley.edu
d
Institute of Urban and Regional Development, University of California,
Berkeley, CA, USA .
E-mail: Hfoster@gisc.berkeley.edu
e
Haas School of Business, University of California, Berkeley, CA 94720, USA .
E-mail: karlene@haas.berkeley.edu
*Corresponding author: Department of Civil & Environmental Engineering, 212 McLaughlin Hall,
University of California, Berkeley, CA 94720, USA
Abstract
The fundamental thesis of this paper is that no matter how much physical science and technology are involved in complex systems, no system is ever purely or
solely physical or technical. Certainly no system of which we are aware is purely scientic
or technical in its operation or management. Furthermore, while research on and the
modeling of complex systems usually rely heavily on the consideration of technological
variables and processes, they typically fail to consider the contributions of individual psychological, organizational and contextual factors. This paper argues that we need models
that avoid committing errors of the third kind, solving the wrong problem precisely. The
paper sets out a mechanism for developing models that include contextual as well as
technological variables.
Risk Management (2009) 11, 3043. doi:10.1057/rm.2008.12

Keywords: risk analysis; human factors; organizational factors; geographic
information systems; environmental impact statements; high reliability
organizations
2009 Palgrave Macmillan 1460-3799/09 Risk Management Vol. 11, 1, 3043
www.palgrave-journals.com/rm/
A new approach to risk
Introduction
hat do the Exxon Valdez spill, the Katrina levee failure and flood
and the Piper Alpha Platform failure disasters have in common?
They occurred because of the failure to recognize oil infrastructure,
ship-safety and flood control as complex infrastructure systems (CISs). Such
systems require risk assessments that include psychological, social, organizational and political processes in addition to those typical of traditional engineering practices. As a result, we suggest reformulating the problem of risk. To
give appropriate weight to social processes in risk assessment, we suggest applying findings from other disciplines including agent-based modeling (ABM),
the use of geographic information systems (GISs) to integrate multi-scale and
multi-discipline input, technology delivery system (TDS) design and high reliability organization (HRO) management principles.
The Assessment and Calculation of Risk

In engineering infrastructures that must cope with natural hazards, designers
traditionally calculate risk for two reasons: to prioritize design so that the most
likely and potentially most damaging hazards get the most attention, and to
evaluate the adequacy of design. For example, when a design lowers the threat
of a hazard to a value comparable to other acceptable hazards, that design is
good enough. Risk assessment shapes design, construction and management of
infrastructure systems solutions so great attention needs to be paid to how it is
done.
Risk assessment in complex systems is strongly dependent on five crucial
factors:
1. the inherent complexity of the system and the environment in which it
exists and operates;
2. the models used to represent the system; that is, how the system and its
environment, and hence its complexity, are represented in the first place;
3. whether the models give equal weight to technical, individual human,
organizational and socio-political (for example, legal) variables in determining the operation and the failure modes of the system; for instance,
whether certain variables (for example, engineering or technical) are
emphasized or privileged over others, and whether the representation of the
system is fundamentally biased or flawed to begin with;
4. as a direct result of factor 3, the number and kinds of terms included in
determining the probability, or the probabilities, of failure of the system,
and;
5. how the consequences of the failure of the system are also represented and
determined.
31
Bea et al
The fundamental thesis of this paper is that no matter how much physical
science and technology are involved in a complex system, no system is ever
purely or solely physical or technical. Certainly no system of which we are
aware is purely scientific or technical in its operation or management.
Every system consists of a complex set of (a) technical processes and variables that interact strongly with a complex set of (b) individual human (that is,
psychological), (c) organizational and (d) socio-political processes and variables. Technical, individual, and so on variables that compose the system can
only be distinguished from one another with great difficulty. In other words,
the variables are so strongly coupled that it is almost impossible to determine
where one kind typically begins and others end or leave off.
By its very nature, modeling complex systems is inherently interdisciplinary.
This means that determinations of the probabilities of system failure are also
inherently interdisciplinary. In turn, the assessment of risks associated with
complex systems is inherently interdisciplinary as well.
In spite of this, the modeling and risk assessment of complex systems have
not been as interdisciplinary as they need to be. As a result, a basic and fundamental error underlies the vast majority of risk assessments. This error is
known as the Error of the Third Kind, or the Type Three Error (E3) (Mitroff
and Linstone, 1992).
E3 is defined as the probability of solving the wrong problem precisely.
Whereas Type One (E1) and Type Two (E2) errors are well known and utilized
in statistics, E3 is not. E1 and E2 (accepting or rejecting a null hypothesis)
relate to problems that are already known or well defined. In sharp contrast,
E3 pertains to how problems are defined or formulated in the first place. In this
sense, E3 is both prior to and more basic than E1 and E2.
This paper shows that by taking (a) technical, (b) individual human, (c) organizational and (d) socio-political variables equally into account, E3 can be
expressed on a quantitative basis like E1 and E2. Anything less leads to dangerously misleading risk assessments.
An interdisciplinary approach to modeling complex systems allows us to
formulate and determine the E3s associated with them. Combating E3s in
practice also requires an interdisciplinary approach. Organizations that relegate risk assessment to individuals with narrow technocratic expertise will inevitably commit E3s. Only by incorporating multiple perspectives and being
alert to discrepancies between models and reality can organizations deal with
risk in a realistic way.
Background
Work on this paper started almost two decades ago with an investigation
by one of the authors (Bea) of the dramatic failure of the Piper Alpha offshore
oil and gas drilling and production platform in the North Sea. This platform
32
served as a hub in a major part of the oil and gas infrastructure in the
North Sea. The investigative report stated that the majority of the causes
of this failure (80 per cent or more) were firmly rooted in human, organizational and institutional malfunctions. The remaining causes could reasonably be attributed to malfunctions in the engineered parts of this complex
system. This was a rude awakening because the platform was intensely
studied prior to its failure using traditional engineering approaches and engineering fixes were put in place. However, these fixes proved to be totally
ineffective.
Defining the problem as primarily an engineering problem, commits a major E3. Hence, problem definition is critical in designing, operating, maintaining and managing critical CISs. In the Piper Alpha situation a new problem
was exposed that involved other parts of this production infrastructure. When
the first fires and explosions erupted on the platform, personnel on interconnected production platforms realized that the pressures in the pipelines had
dropped. In response to the drop in pipeline pressure and organizational pressures to catch up on back production, these platforms increased production
to the Piper Alpha platform, further escalating and accelerating the final melt
down of the system.
It was subsequently recognized that a broader, more holistic problem definition is of critical importance in designing, operating, maintaining and managing CISs. Findings such as this are now common in investigations of other
disasters (for example, Challenger and Columbia, Texas City and Bhopal,
Katrina and Betsy, and so on). Most recently, this background was incorporated into an NSF-funded research project to investigate the causes of the failure of the flood defense system for the Greater New Orleans Area (Kardon
et al, 2006; Seed et al, 2007ac).
The human, organizational and institutional causes are termed extrinsic.
The categories of uncertainties traditionally addressed by engineers natural
or inherent (aleatory) and those associated with parametric, state and analytical model uncertainties (epistemic) are termed intrinsic. Because the neglected
extrinsic factors are actually fundamental to system performance, expected
risks were under-predicted by factors of 100 or more. These findings are consistent with a large body of research that highlights the role of extrinsic factors in large-scale system failures (for example, Perrow, 1984; Roberts, 1990;
Clarke and Short, 1993; Vaughan, 1996, 1999).
Traditional engineering analyses and processes also result in inappropriate
strategies for managing risk. Another example of an E3 that is the result
of thinking that overemphasizes improving things such as system components, rather than addressing process and people factors that produce risk
and the consequences of risk. Compelling evidence for this is available in reports of major catastrophes such as Bhopal (Shrivastava, 1987), Columbia
(Gehman et al, 2003) and Katrina (Farber et al, 2007).
33
Bea et al
Figure 1:
Evaluating and managing CISs risks.
A Proposal for Studying Complex Systems

This paper proposes a new approach to developing a holistic approach to understanding and managing risks and their consequences associated with CIS
failures. As shown in Figure 1, this new approach incorporates analytic methods that model relationships among factors and processes taking place at four
levels of analysis: physical systems, organizational processes and practices, and
the broader societal context.
Level 1, physical systems and their components, is the domain of traditional
engineering risk analysis and management. Level 2 includes human elements of
organizations traditionally studied by psychologists. These include individual
differences, personality, training, and so on. Scholars specializing in the sociology of organizations, management science, organizational communication and
related fields traditionally study level 3, which encompasses organizational attributes and processes. Included in this level is a range of factors, including
organizational structure, culture, management and problem-identification, and
problem-solving strategies. Level 4 incorporates broader societal factors that
affect both organizational processes and the physical elements of CISs. This
level consists of more macro-level factors such as governance, laws and regulatory regimes, and social, demographic and economic forces that must also be
taken into account in CISs risk and vulnerability analyses.
34
Often level 1 analyses fail to address the critically important issues associated with the consequences of failure particularly those associated with rescue and recovery resilience. Levels 2, 3 and 4 are the important additional
elements contributed by individual differences psychology, organizational and
social sciences to enable a more holistic assessment of risks and the management alternatives that are available to reduce the likelihoods of failures
and consequences contributing to the CISs risks (Roberts and Sloane, 1988;
Roberts et al, 2004, 2005).
The guiding logic of our approach is that a full understanding of CIS vulnerability can only be achieved through the analysis of interactions within and
across these four levels, in context and over time. As discussed above, prior
engineering research has focused on the first level the physical elements that
make up engineered systems while treating the other two levels as extrinsic
to formal analytic frameworks. In contrast, this paper recognizes that managing risks associated with CISs is a multi-dimensional problem that must be
addressed through collaborative research and educational activities that cross
and transcend disciplinary boundaries.
An Approach to Assessing Risks Associated with CISs

The probability of failure, P(F), of a CIS is
P ( F ) = P ( FI FE )
(1)
where I stands for intrinsic factors, E stands for extrinsic factors and stands
for the Union operator. I typically stands for technical factors such as the failure of levees and pumping systems, while E stands for organizational/social
factors such as the breakdown of communications between different entities
charged with managing a CIS.
In turn,
P ( F ) = P ( FI / E )P ( E ) + P ( FI / Not E )P ( Not E )
+ P ( FE / E )P ( E )
(2)
The first term in equation (2) addresses the likelihood of system failure due
to intrinsic factors (technical) given (that is, conditional upon) the uncertainties associated with extrinsic factors (psychological, organizational, social,
legal, and so on). The second term addresses the same likelihood given no
extrinsic factors. By our initial assumption that every complex system is composed of the interactions between technical and social variables, the second
term is impossible. We include it, nonetheless, for an important reason that
will become apparent shortly. The third term addresses the likelihood of
system failures due directly to extrinsic factors.
35
Bea et al
Equation (2) leads to an interesting and important way to measure E3.

Recall that E3 is the probability of solving the wrong problem precisely. This
can be expressed as follows in equation (3)
P[P ( F )] = P[P ( FI / E )P ( E )
+ P ( FI / Not E )P ( Not E )]
(3)
P[P(F)] is a probability distribution/function like any other probability

distribution/function. It is the probability that the probability of failure
function only includes the first two terms. That is, P(P(F)) is a way to measure
whether assessing the probability of failure of a complex system is solving the
wrong problem through the use of the wrong (that is, incomplete) formula.
The Practical Signicance of E3

E3 is critically important in understanding system failures. As noted earlier,
work relevant to this article started almost two decades ago with a study of an
oil platform failure. This experience led to researcher involvement in investigations of other failures of engineered systems including the Exxon Valdez, the
Columbia space shuttle, the Texas City BP refinery and the flood protection
system for the Greater New Orleans area (Kardon et al, 2006; Bea, 2007a, b;
Farber et al, 2007; Seed et al, 2007ac). The theme developed from these
experiences was that the majority (80 per cent or more) of the causes of
failures were humanorganizationalinstitutional in nature. These causes are
termed extrinsic. The balance of the causes of failure can be traced to two
categories of uncertainties traditionally addressed by engineers natural or
inherent (aleatory) and those associated with parametric, state and analytical
model uncertainties (epistemic). These causes are termed Intrinsic.
This was an important finding because it helped to explain why traditional
engineering analyses of the likelihoods of failures do not match the actual or
actuarial likelihoods of failure they under-predict the real likelihoods by factors of 10 or more. Engineering models do not include the critical human and
organizational parts of the system resulting in a critical E3. A similar situation also was found with the consequences of failure these too were underpredicted by factors of 10 or more. Thus, expected risks taken as the product
of the likelihood of failure and the consequences given failure were underpredicted by factors of 100 or more.
Traditional engineering analyses and processes result in distorted approaches to better manage risks (combination of likelihoods and consequences
of failures). Again, another major E3. Frequently, attempts are made to fix
things rather than processes and people. Traditional approaches focus on
proactive assessments and management strategies. But, experience with these
36
failures clearly indicates there are important limitations to proactive assessments and the associated management strategies. The future changes things;
systems are more organic than mechanical; and predictability is extremely limited. Even reactive (after the accident or failure) analyses and associated
approaches are limited because they focus on things not on processes and
people. This leads to trying to fix the wrong things in the wrong ways.
Ways to Deal with E3

A major cause of E3s is that key portions of interactive systems particularly
the soft human and organizational portions are omitted from analysis in
part because of the absence of rigorous modeling methodologies. ABM is a
promising method for addressing these issues (Gilbert and Terna, 2000;
Cummings et al, 2006; Axelrod and Tesfatsion, 2007). ABM is a specific
simulation technique that models complex adaptive systems via computergenerated agents that interact in a virtual environment. These agents can
represent individual people, but they can also represent social groupings such
as operating teams, organizations, firms, communities and agencies. The interactions occur according to representative programmed behavioral rules that
create the unpredictable self-organizing behavior seen in complex adaptive
systems. The behavioral rules are informed by case studies, observations of
CISs operations and expert judgment.
GISs provide another important modeling tool. GISs have long been used to
store, manipulate and display spatial data. In addition to their obvious utility
in managing environmental data, they allow designers to encode solutions so
they can be evaluated and compared with each other quantitatively in terms of
whatever measures are determined to be useful. In addition, because a GIS allows the display of concepts and relationships in map form to large audiences,
it is the ideal tool for integrating traditional engineering and social science
analyses. GISs can serve as a monitoring tool to integrate sensor data, field
reports, remote sensing data, and so on, so system management can be integrated with design solutions. Finally, for managing complex systems, generalization algorithms (Radke and Mulan, 2000, Radke et al, 2000) aggregate
observational data so that broad trends can be recognized and responded to.
A key objective in this research is to create and validate methods and procedures to enable meaningful characterizations and quantifications of P(E). However, quantifications are not the primary goal. The primary goal is to develop
insights into how P(E) can be reduced by improving the process and people
aspects of CISs. The quantifications provide metrics to assist evaluations of
alternatives and progress toward improving the quality and reliability of CISs.
Ultimately, we need better delivery of Risk Assessment and Management
Infrastructure Systems technology. Some preliminary work was done to design
an advanced TDS (Bea, 2007b). This work resulted in identification of three
37
Bea et al
interrelated components: (1) the public/s (people affected by the CISs), (2) the
governments (of, by and for the people with responsibilities for the CISs) and
(3) industry (responsible for providing CISs). The linkages among these components are facilitated and enhanced with modern communication and information technology including the media and GISs. The fundamental objective
is to provide improved information and knowledge that will help impact values, beliefs and behaviors in ways beneficial to the publics and to the environments in which they exist. At present the concepts associated with the TDS are
used in efforts to integrate flood protection strategies and procedures into
improving the flood protection systems for the Greater New Orleans and
Sacramento Delta areas.
Developing effective TDSs is one of the most critical parts of building resilient and sustainable CISs. Without the required societal and political wills,
the technology ways to improve resilience, sustainability and reliability of
CISs will not be effectively implemented.
For the last 20 years research on HROs examined a number of adaptive
management strategies that work to render organizations highly reliable and
sustainable. One finding suggests that adaptable organizations change their
structures in response to changing conditions. When their environments are
very uncertain HROs flatten their structures considerably, returning to more
hierarchical structures as their environments gain more certainty. Another
characteristic of HROs is that they push decision making to the lowest level of
the organization commensurate with the knowledge needed to make that decision. In other words, if a decision about refueling an aircraft in the fast paced
and potentially dangerous environment of an aircraft carrier is best made by a
chief petty officer on the deck, it is certainly not given over to the ships captain
on the bridge of the ship (Weick and Roberts, 2003). These kinds of structural
and decision-making strategies render the organization more resilient than are
organizations who do not follow them. This resilience opens the organization
up to the possibilities of looking for potential E3s and doing something to
correct the situation.
It is hypothesized that the adaptable CISs do much the same thing. A good
deal of networking research has been done in organizational behavior. An initial step in understanding how CISs adapt and make decisions is to uncover
their networks of relationships. It is hypothesized that more resilient CISs have
more tentacles into other complex systems than less resilient CISs. Other aspects of the influence of both political decisions and organizational processes
need to be included in dealing with CISs.
Engineers are trained to focus on technical errors. Narrow and exclusive
focus on technical factors is a source of E3s, simply because engineers tend to
place too much reliance on technical models without realizing the likelihood
that those models fail to capture key elements of risk. If engineers and other
system designers can learn to take a broader perspective, E3s can be reduced.
38
Nevertheless, even enlightened technical designers inevitably have limited

perspectives, based on their own training and limited sources of information.
Minimizing E3s requires opening the planning process to those with other
perspectives, including natural and social scientists. The planning process also
needs to include individuals with on the ground experience with the system in
question. Thus, what is frequently a closed technocratic planning process must
become much more open and public.
A More Open and Public Perspective

Ideally, the environmental assessment procedure can provide one path toward
this expanded planning process. Major infrastructure projects typically involve
participation by government decision makers in either funding or licensing.
The planning process used by these decision makers makes some effort to consider issues of resilience and sustainability, as well as potential interactions
among infrastructures. A primary tool for considering these issues is environmental assessment. These assessments take the form of environmental impact
statements (EISs) or environmental impact reviews (EIRs) (Gerschwer, 1993).
One part of creating better decision tools for infrastructure is understanding
the role of environmental assessment in current planning efforts. Understanding what works and does not work (attempting to avoid E3s) creates the
opportunity for improved methodologies. Criticisms of environmental assessments provide rich research issues (Klick, 1994). Two relevant criticisms are
that the process places undue confidence in predictions and too little emphasis
on monitoring and adaptive management. In addition, consideration of interaction between projects is handicapped by a series of Supreme Court decisions
(Karkkainen et al, 2000).
Despite the inadequacies of current environmental assessment, its aspirations are consistent with the kind of system analysis needed to avoid E3s. The
National Environmental Quality Act (NEPA) directs all federal agencies to
engage in systematic, interdisciplinary approaches that include integrated use
of the natural and social science and the environmental design arts (West
Publishing Co., 2008). It also requires agencies to recognize that environmental issues are worldwide and long-range and where consistent with US foreign
policy to maximize international cooperation in dealing with the decline in
the quality of mankinds world environment (West Publishing Co., 2008). The
environmental assessment process also includes provisions designed to open
the process to multiple perspectives. Public notice and the opportunity to
submit written comments are routine. Perhaps more importantly, agencies are
required to engage in consulting other agencies, many of which have different
goals and perspectives that can be critical in identifying E3s. Too often project
designers view environmental review as an irksome constraint on their planning, rather than recognizing it as an opportunity to avoid critical E3s.
39
Bea et al
GIS can provide a methodology for the kind of broad-gauged planning process needed to minimize E3s. For example, one use of GIS for environmental
assessment broke the geographical area into cells of areas with similar vegetation, climate and soils. A model was used to predict, on a cell-by-cell basis, the
growth and aging of a forest, including the size and distribution of each forest
type. Those calculations in turn were used together with a habitat suitability
model to predict impacts on wildlife (Eady, 1995). In another instance, the
Bureau of Reclamation made good use of GIS in performing an assessment of
the operations of the Glen Canyon Dam. Public interest was very high, with
more than 30 000 people commenting on the draft of the environmental EIS.
Thus, GIS contributed significantly to the planning process, both in terms of
procedure and in terms of allowing a broad synthetic analysis, as the White
House Council on Environmental Quality (1997) explained:
GIS provides the analyst with management of large data sets, data overlay and
analysis of development and natural resource patterns, trends analysis, mathematical impact modeling with locational data, habitat analysis, aesthetic analysis, and
improved public consultation. Using GIS has the potential to facilitate the efficient
completion of projects while building confidence in the NEPA process.
We also need to consider the incentives that will lead system designers to
broaden their horizons and augment the planning process. One such mechanism is the potential for civil liability. The potential for liability can push designers to consider broader ranges of risk. Similarly, insurance companies can
play a proactive role in encouraging safe design, bringing to bear their broad
range of experience with other system failures and safety methodologies.
In seeking to avoid E3s, we can also benefit from the rich literature about
organizational learning. Organizations learn by embedding historical experience
in their routines (Levitt and March, 1988). Organizational routines are based on
implicit models that help the organization make sense of the world and respond
to perceived problems. These models are as subject to E3 as are the more formal
engineering models. However, without conditions motivating change, routines
are often relatively stable and organizations generally tend to be inert, relying on
existing models and adapting less than perfectly to and falling in and out of
alignment with their environments (Nelson and Winter, 1982). Disaster preparation calls for a different form of learning in which organizations draw on not
only their own experiences but also those of other organizations. Such network
effects exist for a variety of learning processes (for example, Argote et al, 1990;
Baum and Ingram, 1998; Beckman and Haunschild, 2002).
HROs are also concerned with learning. They are careful to accept input
from individuals at all levels of the organization, thereby broadening their
base of knowledge and perspectives, and they pay careful attention to unexpected outcomes and system failures (Roberts, 1990; Weick and Roberts,
40
2003). Thus, they are able to detect the shortcomings of their implicit models
and avoid E3s.
Over the past few decades, scholars from many disciplines have advocated relational or systems approaches, as opposed to reductionist approaches that study
particular events and entities in isolation (Miller, 1972; Wolf, 1980). For instance,
collaborative governance involving multiple organizations both public and private is a principal focus in recent environmental and administrative law scholarship (Freeman, 1997; Minow, 2003). We are gaining solid information about how
these interactions work in the context of regulation (Freeman, 1997; Cunningham
et al, 2003), and in developing policy networks (Agranoff, 2003). Researchers are
beginning to understand how law can facilitate formal and informal relations that
achieve the appropriate balance between accountability to public goals, and flexibility necessary for maximizing the utility of private-sector involvement (Karkkainen et al, 2000; Bamberger, 2006).
Conclusion
All too often, researchers and decision makers focus exclusively on E1s, the risk
of accepting a false hypothesis about the true value of a variable. They fail to
take into account E2s, the risk of rejecting a true hypothesis about the true value
of a variable. Thus, statistical reliability trumps statistical power. But even more
important are E3s the risk that the entire model used in the analysis is wrong,
often because it omits key variables. For researchers, this can be merely a methodological headache, which goes under the name of specification error or omitted variables bias. But for decision makers, the consequences can be literally
deadly. Models can produce precise calculations of the value of a risk that are
nonetheless meaningless because the model is radically incomplete.
In this paper, we attempted to propose methodologies for dealing with E3s in
risk assessment. As we saw, E3s are to some extent subject to rigorous
analysis, and promising methodologies exist with which to improve formal modeling. But the greater challenge may be to design human systems for risk analysis
that allow E3s to be detected and corrected. Such systems require broad input
and a willingness to reassess models in light of the unexpected. In designing such
systems of risk assessment, we must both improve formal modeling and learn
from the organization literature to design better processes for decision-making.
Acknowledgement
This project was supported, in large part, by the National Science Foundation
(NSF) under EFGRI Grant No. 0836047. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors
and do not necessarily reflect the views of NSF.
41
Bea et al
Re ference s
Agranoff, R. (2003) A New Look at the Value-adding Functions of Intergovernmental
Networks. Paper presented for National Public Management Research Conference,
Georgetown University, Washington DC.
Argote, L., Beckman, S.L. and Epple, D. (1990) The persistence and transfer of learning
in industrial settings. Management Science 36(2): 140154.
Axelrod, R. and Tesfatsion, L. (2007) On-line guide for newcomers to agent based
modeling in the social sciences, www.econ.iastate,edu/testfatsi/abmread.htm, accessed
5 January 2007.
Bamberger, K.A. (2006) Blurring Boundaries: Organizational Theory, Regulated Firms,
and the Administrative State. Berkeley: University of California. Working paper.
Beckman, C.M. and Haunschild, P.R. (2002) Network learning: The effects of partners
heterogeneity of experience on corporate acquisitions. Administrative Science Quarterly 47(1): 92124.
Baum, J.A.C. and Ingram, P. (1998) Survival-enhancing learning in the Manhattan hotel
industry, 18981980. Management Science 44(7): 9961016.
Bea, R.G. (2007a) Reliability Assessment and Management Lessons from Hurricane
Katrina, OMAE 2007-29650, Proceedings of the Sixth International Conference
on Offshore Mechanics and Arctic Engineering, New York: American Society of
Mechanical Engineers.
Bea, R.G. (2007b) Lessons From Failure of the Flood Protection System for the Greater
New Orleans Area During Hurricane Katrina, OMAE 2007-29649, Proceedings of
the Sixth International Conference on Offshore Mechanics and Arctic Engineering,
New York: American Society of Mechanical Engineers.
Clarke, L. and Short, J. (1993) Social organization and risk: Some current controversies.
Annual Review of Sociology 19: 375399.
Cummings, M.C., McGarvey, D.C., Vinch, P.M. and Colletti, B.W. (2006) Homeland
Security Risk Assessment, RP05-024-01a, Arlington, VA: Homeland Security Institute.
Cunningham, N., Kagan, R.A. and Thornton, D. (2003) Shades of Green: Business,
Regulation, and Environment. Stanford, CA: Stanford University Press.
Eady, W. (1995) The use of GIS in environmental assessment. Impact Assessment
13: 199206.
Farber, D.A., Bea, R.G., Roberts, K., Wenk, E. and Inkabi, K. (2007) Reinventing flood
control. Tulane Law Review 81(4): 10851127.
Freeman, J. (1997) Collaborative governance in the administrative state. UCLA Law
Review 45(1): 199.
Gehman, H.W. Jr. et al. (2003) Columbia Accident Investigation Report, Vols. 6,
Washington DC: Government Printing Office.
Gerschwer, L. (1993) Informational standing under NEPA: Justiciability and environmental decisionmaking process. Columbia University Law Review 93: 9961001.
Gilbert, N. and Terna, P. (2000) How to build and use agent-based models in social
science. Mind and Society 1: 5772.
Miller, J.G. (1972) Living Systems. New York: McGraw Hill.
Minow, M. (2003) Partners, Not Rivals: Privatization and the Public Good. Boston, MA:
Beacon Press.
Kardon, J.B., Bea, R.G. and Williamson, R.B. (2006) Validity and Reliability of Forensic
Engineering Methods and Processes. Herndon, VA: American Society of Civil Engineers.
Karkkainen, B.C., Fung, A. and Sabel, C. (2000) After backyard environmentalism:
Toward a performance-based regime of environmental regulation. American Behavioral Scientist 44(4): 690709.
42
Klick, K.A. (1994) The extraterritorial reach of NEPAs EIS requirement after environmental defense fund v. Massey. American University Law Review 44: 291322.
Levitt, B. and March, J.G. (1988) Organizational learning. Annual Review of Sociology
14(2): 319340.
Mitroff, I.I. and Linstone, H. (1992) The Unbounded Mind. New York: Oxford
University Press.
Nelson, R.R. and Winter, S.G. (1982) An Evolutionary Theory of Economic Change.
Cambridge, MA: Harvard University Press.
Perrow, C. (1984) Normal Accidents: Living with High Risk Technologies. New York:
Basic Books.
Radke, J. and Lan, M. (2000) Spatial decompositions, modeling and mapping service
regions to predict access to social programs. Geographic Information Sciences 6(2):
105112.
Radke, J.T., Cova, M.F., Sheridan, M., Troy, A., Lan, M. and Johnson, R. (2000)
Application challenges for GIS science: Implications for research education, and
policy for risk assessment. Emergency Preparedness and Response (RAEPR)? URISA
Journal 12(2): 1530.
Roberts, K.H. (1990) Some characteristics of one type of high reliability organization.
Organization Science 1(2): 160176.
Roberts, K.H. and Sloane, S.B. (1988) An Aggregation Problem and Organizational
Effectiveness. In: B. Schneider and D. Schoorman (eds.) Facilitating Organizational
Effectiveness. Lexington, MA: Lexington Press, pp. 125144.
Roberts, K.H., Madsen, P. and Desai, V. (2004) Bridging Levels, Variables, and
Methodologies. In: F.J. Yammarino and A.E. Dansereau (eds.) Research in Multi
Level Issues: An Annual Series. Oxford, UK: Elsevier, pp. 6978 (also in Science Direct).
Roberts, K.H., Madsen, P. and Desai, V. (2005) The Space Between in Space Transportation: A Relational Analysis of the Failure of STS 107. In: M. Farjoum and W. Starbuck
(eds.) Organizations at the Limit: Lessons from the Columbia Disaster. Malden, MA:
Blackwell Publishing, pp. 8198.
Seed., R.B. et al. (2007a) Investigation of Levee Performance in Hurricane Katrina:
The New Orleans Drainage Canals, Proceedings Geo, Denver, 2007, ASCE.
Seed, R.B. et al. (2007b) Investigation of Levee Performance in Hurricane Katrina:
The Inner Harbor Navigation Canal, Proceedings Geo, Denver, 2007, ASCE.
Seed, R.B. et al. (2007c) Investigation of the Performance of the New Orleans Regional
Flood Protection Systems During Hurricane Katrina: Lessons Learned, Proceedings
Geo, Denver, 2007 ASCE, pp. 116.
Shrivastava, P. (1987) Bhopal: Anatomy of a Crisis. Cambridge, MA: Ballinger.
Vaughan, D. (1996) The Challenger Launch Decision: Risky Technology, Culture, and
Deviance. Chicago, IL: University of Chicago Press.
Vaughan, D. (1999) The dark side of organizations: Mistake, misconduct, and disaster.
Annual Review of Sociology 25: 271305.
Weick, K.E. and Roberts, K.H. (2003) Collective mind in organizations: Heedful
interrelating on flight decks. Administrative Science Quarterly 38: 357381.
West Publishing Co. (2008) Selected Environmental Statutes 20082009, Educational
edn. St. Paul, MN.
White House Council on Environmental Quality. (1997) The National Environmental
Policy Act: A Study of its Effectiveness after 25 Years, http://www.nepa.gov/nepa/
nepa25fn.pdf.
Wolf, F.A. (1980) Taking the Quantum Leap. New York: Harper and Row.
43
GARP Code
of Conduct
Adopted: February 26, 2007

Revised: June 23, 2010
June 23, 2010
I.
]
Risk practitioners should understand these as concepts
Introductory Statement
that reflect an evolving shared body of professional stanThe GARP Code of Conduct (Code) sets forth principles
dards and practices. In considering the issues this raises,
of professional conduct for Global Association of Risk
ethical behavior must weigh the circumstances and the
Professionals (GARP), Financial Risk Management (FRM)
culture of the applicable global community in which the
and Energy Risk Professional (ERP) certifications and other
practitioner resides.
GARP certification and diploma holders and candidates,

GARPs Board of Trustees, its Regional Directors, GARP
II.
Code of Conduct
Committee Members and GARPs staff (hereinafter collectively referred to as GARP Members) in support of the
The Code is comprised of the following Principles, Pro-
advancement of the financial risk management profession.
fessional Standards and Rules of Conduct which GARP
These principles promote the highest levels of ethical con-
Members agree to uphold and implement.
duct and disclosure and provide direction and support for

both the individual practitioner and the risk management
1.
Principles
1.1
Professional Integrity and Ethical Conduct. GARP
profession.
The pursuit of high ethical standards goes beyond
following the letter of applicable rules and regulations and
Members shall act with honesty, integrity, and compe-
behaving in accordance with the intentions of those laws
tence to fulfill the risk professionals responsibilities
and regulations, it is about
and to uphold the reputation of the risk management
The pursuit of high ethical
pursuing a universal ethical
profession. GARP Members must avoid disguised con-
standards goes beyond following
culture.
trivances in assessments, measurements and processes
the letter of applicable rules
All individuals, firms
that are intended to provide business advantage at the
and regulations and behaving
and associations have an
in accordance with the intentions
ethical character. Some of
of those laws and regulations,
the biggest risks faced by
it is about pursuing a universal
firms today do not involve
sibility to promote the interests of all relevant con-
ethical culture.
legal or compliance viola-
stituencies and will not knowingly perform risk
tions but rest on decisions
management services directly or indirectly involving
involving ethical considerations and the application of
expense of honesty and truthfulness.

1.2 Conflicts of Interest. GARP Members have a respon-
an actual or potential conflict of interest unless full
appropriate standards of conduct to business decision
disclosure has been provided to all affected parties
making.
of any actual or apparent conflict of interest. Where
There is no single prescriptive ethical standard that can

be globally applied. We can only expect that GARP Mem-
conflicts are unavoidable GARP Members commit to

their full disclosure and management.
bers will continuously consider ethical issues and adjust their

conduct accordingly as they engage in their daily activities.
This document makes references to professional standards and generally accepted risk management practices.
1.3 Confidentiality. GARP Members will take all reasonable

precautionary measures to prevent intentional and unintentional disclosure of confidential information.
June 23, 2010
2.
consideration that could be reasonably expected to
Professional Standards
compromise their own or anothers independence and

objectivity.
2.1 Fundamental Responsibilities.
GARP Members must endeavor, and encourage

others, to operate at the highest level of profes-
1.3 Must take reasonable precautions to ensure that the
sional skill.
Members services are not used for improper, fraudulent
GARP Members should always continue to perfect
or illegal purposes.
GARP Members have a personal ethical respons-
their expertise.
analysis, recommendations, actions, or other profes-
responsibility to others.
sional activities.
2.2 Best Practices.
1.4 Shall not knowingly misrepresent details relating to
ibility and cannot out-source or delegate that
1.5 Shall not engage in any professional conduct involving
GARP Members will promote and adhere to applica-
dishonesty or deception or engage in any act that
ble best practice standards, and will ensure that
reflects negatively on their integrity, character, trust-
risk management activities performed under his/her
worthiness, or professional ability or on the risk
direct supervision or management satisfies these
management profession.
applicable standards.
GARP Members recognize that risk management
1.6 Shall not engage in any conduct or commit any act
does not exist in a vacuum. GARP Members commit
that compromises the integrity of GARP, the (Financial
to considering the wider impact of their assess-
Risk Manager) FRM designation or the integrity or
ments and actions on their colleagues and the wider
validity of the examinations leading to the award of the
community and environment in which they work.
right to use the FRM designation or any other credentials that may be offered by GARP.
2.3 Communication and Disclosure. GARP Members

issuing any communications on behalf of their firm will
1.7 Shall endeavor to be mindful of cultural differences
ensure that the communications are clear, appropriate
regarding ethical behavior and customs, and to avoid
to the circumstances and their intended audience, and
any actions that are, or may have the appearance of
satisfy applicable standards of conduct.
being unethical according to local customs. If there

appears to be a conflict or overlap of standards, the
GARP member should always seek to apply the higher
III. Rules of Conduct
standard.
1.
Professional Integrity and Ethical Conduct

2.
Conflict of Interest
GARP Members:
1.1
Shall act professionally, ethically and with integrity

in all dealings with employers, existing or potential
clients, the public, and other practitioners in the
GARP Members:
2.1 Shall act fairly in all situations and must fully disclose
any actual or potential conflict to all affected parties.
financial services industry.

2.2 Shall make full and fair disclosure of all matters that
1.2 Shall exercise reasonable judgment in the provision
could reasonably be expected to impair their independ-
of risk services while maintaining independence of
ence and objectivity or interfere with their respective
thought and direction. GARP Members must not offer,
duties to their employer, clients, and prospective clients.
solicit, or accept any gift, benefit, compensation, or
June 23, 2010
3.
Confidentiality
GARP Members:
3.1 Shall not make use of confidential information for
5.
General Accepted Practices

GARP Members:
5.1 Shall execute all services with diligence and perform all
inappropriate purposes and unless having received
work in a manner that is independent from interested
prior consent shall maintain the confidentiality of their
parties. GARP Members should collect, analyze and
work, their employer or client.
distribute risk information with the highest level of

professional objectivity.
3.2 Must not use confidential information to benefit

personally.
5.2 Shall be familiar with current generally accepted risk

management practices and shall clearly indicate any
4.
Fundamental Responsibilities
GARP Members:
4.1 Shall comply with all applicable laws, rules, and
departure from their use.

5.3 Shall ensure that communications include factual data
and do not contain false information.
regulations (including this Code) governing the GARP

Members professional activities and shall not knowingly
participate or assist in any violation of such laws, rules,
5.4 Shall make a distinction between fact and opinion in

the presentation of analysis and recommendations.
or regulations.
IV. Applicability and Enforcement
4.2 Shall have ethical responsibilities and cannot out-source
or delegate those responsibilities to others.
Every GARP Member should know and abide by this Code.

Local laws and regulations may also impose obligations on
4.3 Shall understand the needs and complexity of their

employer or client, and should provide appropriate and
suitable risk management services and advice.
GARP Members. Where local requirements conflict with the

Code, such requirements will have precedence.
Violation(s) of this Code by may result in, among other
things, the temporary suspension or permanent removal
4.4 Shall be diligent about not overstating the accuracy or

certainty of results or conclusions.
of the GARP Member from GARPs Membership roles, and

may also include temporarily or permanently removing
from the violator the right to use or refer to having earned
4.5 Shall clearly disclose the relevant limits of their specific
the FRM designation or any other GARP granted designa-
knowledge and expertise concerning risk assessment,
tion, following a formal determination that such a violation
industry practices and applicable laws and regulations.
has occurred.
Creating a culture of
risk awareness.TM
Global Association of
Risk Professionals
111 Town Square Place
Suite 1215
Jersey City, New Jersey 07310
USA
+ 1 201.719.7210
Minster House, 1st Floor
42 Mincing Lane
London EC3R 7AE
UK
+ 44 (0) 20 7397 9630
www.garp.org
About GARP | The Global Association of Risk Professionals (GARP) is a not-for-profit global membership organization dedicated to
preparing professionals and organizations to make better informed risk decisions. Membership represents over 100,000 risk management practitioners and researchers from banks, investment management firms, government agencies, academic institutions, and
corporations from more than 195 countries. GARP administers the Financial Risk Manager (FRM) and the Energy Risk Professional
(ERP) exams; certifications recognized by risk professionals worldwide. GARP also helps advance the role of risk management via
comprehensive professional education and training for professionals of all levels. www.garp.org.
2010 Global Association of Risk Professionals. All rights reserved. 6-10

Enterprise Risk Management and Ethics Readings PDF

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Enterprise Risk Management and Ethics Readings PDF

Transféré par

Droits d'auteur :

Formats disponibles

2014

Energy Risk Professional

READINGS THAT ARE FREELY AVAILABLE ON THE GARP WEBSITE

Readings for Enterprise Risk Management and Business Ethics

COSO, Understanding and Communicating Risk Appetite. (January 2012).

Describe the objective and characteristics of an effective risk appetite statement.

COSO, Risk Assessment in Practice. (October 2012).

Describe the steps in the risk assessment process.

Compare and contrast qualitative and quantitative measurement techniques in

2014 Global Association of Risk Professionals. All rights reserved.

Energy Risk Professional

COSO, Developing Key Risk Indicators to Strengthen Enterprise Risk Management.

Describe some qualities of effective KRIs.

Explain how a firm can gather information used in developing KRIs.

Describe how KRIs can be reported, monitored, updated and communicated.

Explain ways that KRIs can provide value to an organization.

Describe the factors used to assess risk in a complex organization or system.

Understand the interdisciplinary aspect of modeling risk associated with a complex

Identify and understand the implications of a Type Three Error (E3).

Summarize the elements used in a Complex Infrastructure System (CIS) risk

GARP Code of Conduct

Global Association of Risk Professionals (GARP). Code of Conduct.

Describe the responsibility of each GARP member with respect to professional

Describe the potential consequences of violating the GARP Code of Conduct.

2014 Global Association of Risk Professionals. All rights reserved.

Thought Leadership in ERM

COSO Board Members

Mark S. Beasley/Douglas F. Prawitt

American Accounting Association (AAA)

American Institute of CPAs (AICPA)

Financial Executives International (FEI)

The Institute of Management Accountants (IMA)

The Institute of Internal Auditors (IIA)

Committee of Sponsoring Organizations

Thought Leadership in ERM

Co m m i t te e o f S p o n s o r i n g Organizations of the Treadway Commission

Risk Appetite Statements

Risk Appetite and Risk Tolerance

Communicating Risk Appetite

Monitoring and Updating Risk Appetite

About the Authors

These questions are embodied in the notion of an entitys

while an organization that is risk-averse, with a low appetite

ERM is not isolated from strategy, planning, or day-to-day

To fully embed ERM in an organization, decision makers

Develop Risk Appetite

Developing risk appetite does not mean the organization

Several common approaches are used to communicate

Once risk appetite is communicated, management, with

This is a common question. Its tone implies two things:

guides an organizations infrastructure, supporting

As an organization decides on its objectives and its

influences the organizations attitudes towards risk;

COSO, Enterprise Risk Management Integrated Framework, p. 19.

Considerations Affecting Risk Appetite

Risk appetite is not developed in isolation from other

consider its existing risk profile, not as a determinant of

The current level and distribution of risks across

The amount of risk that the entity is able to

Acceptable level of variation an entity is willing

The attitudes towards growth, risk, and return

There may be other factors to consider as well. Some

However, the board was well aware of the risks, having