Ciia 50741

Current Issues in Auditing American Accounting Association
Volume 8, Issue 1 DOI: 10.2308/ciia-50741

2014
Pages C1C25
COMMENTARY
On the Use of Checklists in Auditing: A
Commentary
J. Efrim Boritz and Lev M. Timoshenko
SUMMARY: Experimental studies concerning fraud (or red flag) checklists often are
interpreted as providing evidence that checklists are dysfunctional because their use
yields results inferior to unaided judgments (Hogan et al. 2008). However, some of the
criticisms leveled against checklists are directed at generic checklists applied by
individual auditors who combine the cues using their own judgment. Based on a review
and synthesis of the literature on the use of checklists in auditing and other fields, we offer
a framework for effective use of checklists that incorporates the nature of the audit task,
checklist design, checklist application, and contextual factors. Our analysis of checklist
research in auditing suggests that improvements to checklist design and to checklist
application methods can make checklists more effective. In particular, with regard to fraud
risk assessments, customizing checklists to fit both client circumstances and the
characteristics of the fraud risk assessment task, along with auditor reliance on formal
cue-combination models rather than on judgmental cue combinations, could make fraud
checklists more effective than extant research implies.
Keywords: checklists; red flag checklists; decision aids; fraud risk assessment.
INTRODUCTION
Checklists are widely used in auditing to ensure compliance with various requirements, such as
completing audit procedures in the appropriate sequence and without omission, collecting or
compiling all relevant materials required to complete an audit file, and supporting judgment tasks by
listing the key questions to be asked and answered in order to reach appropriate judgments or
conclusions. However, there is a widely held view that the use of diagnostic checklists for fraud risk
J. Efrim Boritz is a Professor and Lev M. Timoshenko is a Ph.D. candidate, both at the University of Waterloo.
We acknowledge the helpful comments on an earlier version of this paper provided by Karen Pincus, and the research
assistance of Chris Wong. We gratefully acknowledge the funding provided by the University of Waterloos Center for
Information Integrity and Information Systems Assurance.
Submitted: March 2013
Accepted: February 2014
Published Online: February 2014
assessment in financial audits yields dysfunctional outcomes (e.g., Hogan, Rezaee, Riley, and Velury
2008; Jamal 2008), and this view is spreading to other uses of checklists (e.g., Wheeler and
Arunachalam 2008; Seow 2011). We agree that, when checklists are incomplete, are too long,
contain inappropriate content, or are applied inappropriately, they can result in checklist failure.
However, some interpretations of audit-checklist failures take a relatively narrow perspective and
blame the failure on simply the use of a checklist rather than on weaknesses in the design or
application of the checklist. Abandoning the use of checklists (e.g., red flag checklists) in favor of
unaided judgment on the basis of such a limited perspective may not be the best option for large firms
with multi-location audits that require a degree of standardization and coordination in the performance
of audit tasks. Our reviewof literature on the use of checklists in auditing and other fields suggests that
improvements in checklist design and application can make checklists more effective.
This study is motivated by the contrast between some of the negative views of checklists
expressed in the extant audit literature; in particular, research on the use of red flag checklists to
support fraud risk assessment and a considerable body of evidence on the successful use of
checklists in many other fields including psychology (Fischhoff, Slovic, and Lichtenstein 1978),
aviation (FAA 2007; Turner 2001), nuclear power generation (Carvalho, dos Santos, and Vidal
2005; Carvalho, Vidal, and de Carvalho 2007), and medicine (Pronovost et al. 2006; Haynes et al.
2009; Ely, Graber, and Croskerry 2011). Gawandes (2009) book, The Checklist Manifesto,
summarizes the benefits achieved by using checklists in the medical field. Kahneman, Lovallo, and
Sibony (2011) support Gawandes (2009) endorsement of checklists, suggesting that checklists
can be helpful in many areas, most notably in corporate decision making. In fact, Kahneman et al.
(2011, 54) provide a 12-question checklist designed to protect corporate decision makers against
various defects in thinking, although Kahneman (2011, 249) recognizes that some practitioners
voice concerns about the impersonality of procedures that are guided by statistics and checklists.
We believe that criticisms of checklists in auditing should be reassessed in light of these
observations.
In this article, we review and synthesize the literature on checklists in auditing and other fields
and consider a comprehensive array of factors that affect the use and effectiveness of checklists in
auditing, with a focus on diagnostic judgment-oriented checklists. Raiffa (1968) defines a
judgmental checklist as an approach to aiding decisions that decomposes a judgment into
components that decision makers can assess, and subsequently combine. We provide a
framework for thinking about checklists that recognizes that checklist-based outcomes are a joint
product of the task and task environment, checklist design, checklist application, and contextual
factors. We conclude that appropriately designed and adequately customized checklists, applied
strategically using appropriate cue-combination methods, can be effective.
CHECKLIST USE IN AUDITING AND OTHER FIELDS
In auditing, checklists have long been utilized to assist auditors in completing a variety of tasks
including client acceptance and retention decisions (Bell, Be dard, Johnstone, and Smith 2002),
inherent risk assessments (Boritz, Albuquerque, and Kielstra 1991; Be dard and Graham 2002),
internal control evaluations (Ashton 1974; Trotman, Yetton, and Zimmer 1983; Boritz 1985),
substantive test planning (Blocher, Esposito, and Willingham 1983; Bonner, Libby, and Nelson
1996), and fraud risk assessments (Pincus 1989). In fact, checklists are a part of GAAS, as the
AICPA (2002, 193197) provides lists of red flags related to misstatements arising from fraudulent
financial reporting, as well as misappropriation of assets. Ironically, just as the virtues of checklists
are being extolled in other fields (e.g., Gawande 2009; Kahneman 2011), the use of checklists in
Boritz and Timoshenko C2
Current Issues in Auditing
Volume 8, Issue 1, 2014
auditing is being challenged, especially for tasks such as fraud risk assessment. Results of
experimental studies examining the use of fraud red flag checklists (Pincus 1989; Asare and
Wright 2004; Seow 2009) are interpreted as providing evidence that checklists are harmful rather
than helpful in detecting fraud (Jamal 2008, 103), and that they restrict the auditors generation of
ideas for fraud detection (Hogan et al. 2008, 239240). Also, recent papers question the
effectiveness of checklists for audit and tax professionals, such as Seow (2011) for an internal
control evaluation task, and Wheeler and Arunachalam (2008) for a tax research task. These
authors argue that checklists increase confirmation bias and focus the users attention solely on
cues included in the checklist. Bamber, Carpenter, and Hammersley (2007) suggest that the
importance of red flag checklists is diminished with the introduction of SAS No. 99s (AICPA 2002)
fraud brainstorming requirement. The authors argue that a SAS No. 99 brainstorming session
cannot be readily reduced to a checklist or form (Bamber et al. 2007, 45).
We reviewed all of the audit research that we could identify that involved the use of checklists,
as well as relevant research in medicine and other fields. Overall, our literature review suggests
that the question of checklists effectiveness for various tasks is far from settled. While some
studies find mixed or weak evidence of checklist usefulness (Blocher et al. 1983; Bonner et al.
1996), or report dysfunctional consequences of their application (Pincus 1989; Asare and Wright
2004; Bamber et al. 2007; Wood 2012), others conclude that checklists are useful for either
performing an audit task (Butler 1985; Alon and Dwyer 2010) or for learning (Seow 2009; J. Rose,
McKay, Norman, and A. Rose 2012). Ashton (1974), R. Libby and P. Libby (1989), Bonner et al.
(1996), Eining, Jones, and Loebbecke (1997), Bell and Carcello (2000), A. Rose and J. Rose
(2003), and Marley (2011) all document improved effectiveness related to using checklist-based
aids. Effectiveness gains appear to be stronger for checklist-based audit decision aids that
supplement a checklists knowledge retrieval function with a model for knowledge aggregation.
Nelson and Tan (2005, 54) suggest that decision aids can be used to enhance independence and
focus the expertise of large firms in the key areas for which they are most vulnerable to audit
failure. However, potential improvements may be attenuated by issues associated with auditor
nonreliance on checklists (Boatsman, Moeckel, and Pei 1997). That is, even when the reliability of
a checklist is supported by evidence, auditors may nevertheless overrule it because of a
preference for relying on their own judgments or a wish to avoid particular consequences, such as
exceeding the budget.
We now examine checklist use in other fields to determine whether auditing practice can
benefit from insights provided by research and practice in those fields. In aviationone of the first
industries to adopt checklistschecklists are intended to assist with operational procedures so
that pilots can confirm that omission of any critical procedural step is detected and remediated
before it becomes consequential. Compliance with standards is achieved via error trapping (FAA
2007; Turner 2001), whereby a procedure continues only if all critical errors are intercepted and
addressed. Psychology research identifies checklist applications in many areas, including nuclear
power generation, in which diagnostic judgmental checklists called fault trees can assist
personnel in determining the causes of system breakdowns (Fischhoff et al. 1978).
1
In addition,
1
A fault tree is a judgmental checklist organized as a decision tree. In more precise terms, in a fault tree, a
system failure state is postulated and sequences of more basic faults contributing to this state are laid out in a
systematic manner (U.S. NRC 1981, I-8). Checklists with similar properties to fault trees have been used in
auditing research and practice in connection with expert systems developed in the 1980s and 1990s for
evaluating loan loss provisions, tax provisions, inherent risk assessments, going concern evaluations, and
other diagnostic tasks with embedded decision trees.
Fischhoff et al. (1978) explain that fault trees can help people who deal with complex fallible
systems to better describe and comprehend these systems. Carvalho et al. (2005) and Carvalho et
al. (2007) report that, at nuclear power plants, personnel use both procedural and judgmental
checklists. In plant procedures, the actions of operators are described by flowcharts to illustrate the
sequence of the systems actions, as well as checklists in which operators document the specific
manual actions they executed. According to Carvalho et al. (2005) and Carvalho et al. (2007),
checklists help operators adapt to a high degree of plant automation. Specifically, when they work
with checklists, operators become more aware of plant conditions, because checklists prompt
them to look for information that is not readily available without a stimulus.
In healthcare, checklists initially were adopted to ensure that doctors and nurses in hospitals
completed critical procedures. For example, checklists were implemented in intensive care units
(ICUs) to help prevent post-surgery complications (Pronovost et al. 2006). Further, the introduction
of a 19-item operating room checklist resulted in an impressive 50 percent reduction in surgical
deaths (Haynes et al. 2009). For more information on the use of such checklists in a healthcare
setting, please refer to Gawande (2007) or, for a more comprehensive discussion, Gawande
(2009). Recently, academics and practitioners have raised questions about whether the successful
implementation of procedural checklists can be expanded to judgmental checklists that could be
used for medical diagnoses (Ely et al. 2011; Winters et al. 2009; Winters, Aswani, and Pronovost
2011). Ely et al. (2011) suggest that a combination of checklists possessing different strengths
could help to minimize a number of individual cognitive biases (such as anchoring, availability,
base rate neglect, premature closure, and others; for details refer to Ely et al. [2011]), as well as to
avoid groupthink when a team of doctors makes a diagnosis.
Based on our review and synthesis of the literature on checklists in auditing and other fields,
we developed Table 1, which classifies checklists along two dimensions: (1) generic versus
customized, and (2) procedural versus judgmental. A generic checklist is a standardized list of
common items to be considered, without being tailored to specific circumstances, whereas a
customized checklist is tailored to fit a specific situation (Cowperthwaite 2012). A procedural
checklist provides a basic outline of the work to be performed, including all or most of the important
steps to be carried out, along with their sequence, which the user can modify as required to
accommodate the details of a particular application. In addition to ensuring standardization of
procedures across, for example, audits, geographical locations, and audit team members, and the
completeness and appropriate sequencing of procedures, checklists provide documentation of the
procedures actually performed, which can facilitate review (Anderson 1977, 413). Recent findings
in PCAOB inspection and peer review reports (Gramling and Watson 2009; PCAOB 2013) suggest
that audit documentation remains a major area of concern for regulators, particularly for audits of
accounting estimates (e.g., fair values) and for fraud risk assessment. The use of checklists,
therefore, can provide the additional benefit of fulfilling documentation requirements imposed by
regulatory and standard-setting bodies, such as the PCAOB.
Table 1 serves as a reminder that, when assessing the strengths and weaknesses of
checklists, it is important to distinguish among the various types of checklists represented in the
four cells in Table 1. As stated earlier, the focus of this paper is on judgment-oriented checklists
such as those used in client acceptance and continuance, internal control evaluation, and fraud
risk assessment, summarized in the cells in the bottom half, and especially the lower right hand
corner of Table 1.
Our review of the literature identified factors that influence the effectiveness of judgment
checklists that we have summarized in Figure 1 under four main headings: nature of the task,
T
A
B
L
E
1
C
l
a
s
s
i
c
a
t
i
o
n
o
f
C
h
e
c
k
l
i
s
t
s
C
h
e
c
k
l
i
s
t
T
y
p
e
G
e
n
e
r
i
c
:
C
h
e
c
k
l
i
s
t
s
T
h
a
t
A
r
e
A
p
p
l
i
c
a
b
l
e
t
o
M
u
l
t
i
p
l
e
S
u
b
j
e
c
t
s
i
n
a
V
a
r
i
e
t
y
o
f
S
i
t
u
a
t
i
o
n
s
C
u
s
t
o
m
i
z
e
d
:
C
h
e
c
k
l
i
s
t
s
A
d
o
p
t
e
d
t
o
a
P
a
r
t
i
c
u
l
a
r
S
u
b
j
e
c
t
a
n
d
/
o
r
t
o
S
p
e
c
i
c
C
i
r
c
u
m
s
t
a
n
c
e
s
P
r
o
c
e
d
u
r
a
l
(
a
c
t
i
o
n
a
b
l
e
p
r
o
c
e
d
u
r
e
)
c
h
e
c
k
l
i
s
t
s
d
e
s
i
g
n
e
d
t
o
a
s
s
i
s
t
w
i
t
h
c
o
m
p
l
i
a
n
c
e
w
i
t
h
a
s
p
e
c
i
e
d
s
e
q
u
e
n
c
e
o
f
a
c
t
i
o
n
s
s
u
c
h
a
s
c
o
m
p
l
e
t
i
o
n
o
f
a
u
d
i
t
d
o
c
u
m
e
n
t
a
t
i
o
n
.
F
A
A
c
h
e
c
k
l
i
s
t
s
(
F
A
A
2
0
0
7
;
T
u
r
n
e
r
2
0
0
1
)
N
u
c
l
e
a
r
p
o
w
e
r
p
l
a
n
t
p
r
o
c
e
d
u
r
a
l
c
h
e
c
k
l
i
s
t
s
(
C
a
r
v
a
l
h
o
e
t
a
l
.
2
0
0
5
a
n
d
C
a
r
v
a
l
h
o
e
t
a
l
.
2
0
0
7
)
P
r
o
c
e
d
u
r
a
l
g
e
n
e
r
i
c
a
u
d
i
t
c
h
e
c
k
l
i
s
t
s
(
B
l
o
c
h
e
r
e
t
a
l
.
1
9
8
3
;
M
c
D
a
n
i
e
l
1
9
9
0
)
C
h
e
c
k
l
i
s
t
o
f
c
o
m
m
o
n
p
i
t
f
a
l
l
s
a
n
d
c
o
g
n
i
t
i
v
e
f
o
r
c
i
n
g
f
u
n
c
t
i
o
n
s
f
o
r
a
s
p
e
c
i
c
d
i
s
e
a
s
e
(
E
l
y
e
t
a
l
.
2
0
1
1
)
a
P
r
o
c
e
d
u
r
a
l
c
u
s
t
o
m
i
z
e
d
a
u
d
i
t
c
h
e
c
k
l
i
s
t
s
(
C
o
w
p
e
r
t
h
w
a
i
t
e
2
0
1
2
)
J
u
d
g
m
e
n
t
a
l
(
c
o
g
n
i
t
i
v
e
p
r
o
c
e
d
u
r
e
)
c
h
e
c
k
l
i
s
t
s
d
e
s
i
g
n
e
d
t
o
a
s
s
i
s
t
w
i
t
h
m
a
k
i
n
g
a
j
u
d
g
m
e
n
t
s
u
c
h
a
s
a
c
c
e
p
t
a
n
c
e
o
f
a
n
e
x
i
s
t
i
n
g
o
r
p
o
t
e
n
t
i
a
l
c
l
i
e
n
t
,
i
n
t
e
r
n
a
l
c
o
n
t
r
o
l
e
v
a
l
u
a
t
i
o
n
,
o
r
a
s
s
e
s
s
m
e
n
t
o
f
f
r
a
u
d
r
i
s
k
.
R
e
p
r
o
d
u
c
t
i
v
e
a
p
p
r
o
a
c
h
t
o
d
i
a
g
n
o
s
i
s
c
h
e
c
k
l
i
s
t
(
E
l
y
e
t
a
l
.
2
0
1
1
)
F
a
u
l
t
t
r
e
e
(
F
i
s
c
h
h
o
f
f
e
t
a
l
.
1
9
7
8
)
D
i
f
f
e
r
e
n
t
i
a
l
d
i
a
g
n
o
s
i
s
c
h
e
c
k
l
i
s
t
(
E
l
y
e
t
a
l
.
2
0
1
1
)
b
J
u
d
g
m
e
n
t
a
l
c
u
s
t
o
m
i
z
e
d
a
u
d
i
t
c
h
e
c
k
l
i
s
t
s
(
C
o
w
p
e
r
t
h
w
a
i
t
e
2
0
1
2
)
G
e
n
e
r
i
c
r
e
d
a
g
c
h
e
c
k
l
i
s
t
(
P
i
n
c
u
s
1
9
8
9
;
A
I
C
P
A
2
0
0
2
)
C
u
s
t
o
m
i
z
e
d
r
e
d
a
g
c
h
e
c
k
l
i
s
t
(
B
o
a
t
s
m
a
n
e
t
a
l
.
1
9
9
7
;
B
e
l
l
a
n
d
C
a
r
c
e
l
l
o
2
0
0
0
)
a
C
o
g
n
i
t
i
v
e
f
o
r
c
i
n
g
f
u
n
c
t
i
o
n
c
h
e
c
k
l
i
s
t
s
e
n
s
u
r
e
t
h
a
t
a
c
o
r
r
e
c
t
p
r
o
c
e
d
u
r
e
i
s
f
o
l
l
o
w
e
d
,
a
n
d
/
o
r
p
r
e
v
e
n
t
a
n
u
n
t
o
w
a
r
d
e
v
e
n
t
(
E
l
y
e
t
a
l
.
2
0
1
1
,
3
1
0
)
.
b
D
i
f
f
e
r
e
n
t
i
a
l
d
i
a
g
n
o
s
i
s
c
h
e
c
k
l
i
s
t
s
b
r
i
n
g
t
o
a
t
t
e
n
t
i
o
n
d
i
a
g
n
o
s
e
s
t
h
a
t
s
h
o
u
l
d
n
o
t
b
e
m
i
s
s
e
d
a
n
d
t
h
o
s
e
t
h
a
t
a
r
e
,
i
n
p
r
a
c
t
i
c
e
,
f
r
e
q
u
e
n
t
l
y
m
i
s
s
e
d
(
E
l
y
e
t
a
l
.
2
0
1
1
,
3
1
0
)
.
F
I
G
U
R
E
1
F
a
c
t
o
r
s
A
f
f
e
c
t
i
n
g
t
h
e
E
f
f
e
c
t
i
v
e
n
e
s
s
o
f
a
C
h
e
c
k
l
i
s
t
checklist design, checklist application, and contextual factors. Below, we discuss these in turn,
using fraud risk assessment as an illustrative example of a diagnostic audit task.
NATURE OF THE TASK
The nature of the task determines the need forand should influence the design ofdecision
aids, such as checklists, to enhance task performance. Although we discuss a variety of audit
checklists throughout this paper, in this section we focus on fraud checklists because of the
prominence of their coverage in the literature and the attention paid by regulators and practitioners
to the auditors assessment of and response to fraud risk (McKee 2010; PCAOB 2012). Fraud risk
generally is classified into two main types: financial statement fraud and misappropriation of
assets.
2
Most of the fraud-related auditing literature addresses financial statement fraud, and we
also focus on this type of fraud. One key difference between fraud detection in an audit setting and
tasks in other fields in which checklists are used (e.g., nuclear power generation, aviation, medical
practice) is in the strategic nature of financial reporting fraud. The fraud perpetrator uses deception
to mislead auditors and counter the auditors attempts to detect fraud through the use of red flag
checklists, whereas no such intentional deception is at play in other fields in which checklists are
used.
Economic game theory suggests that the most effective course of action for a
fraud-perpetrating manager is to follow a randomized strategy when perpetrating the fraud (i.e.,
one that makes it difficult to predict where and when the fraud was perpetrated), and that the
auditors optimal response is to randomize the audit strategy to make it more difficult for
fraud-perpetrating managers to guess where and when the auditor might look for fraud (Jamal
2008, 102). However, current risk-based audit practice represents a type of focused (non-
randomized) strategy, which game theory suggests is suboptimal for countering a randomized
fraud strategy, because managers can use the auditors red flag checklists to guess the auditors
strategy and perpetrate fraud in areas that do not appear to be risky. Therefore, proponents of this
view argue that a risk-based audit is more suitable for detecting errors, because they are
unintentional, than fraud, which is deliberate and strategic (Jamal 2008, 102103). Research by
Hoffman and Zimbelman (2009) concludes that the strategic nature of fraud requires strategic
thinking on the part of the auditor, including whether and how red flag checklists are deployed.
We believe that it is difficult for management to implement a randomized fraud strategy
because of the constraining effects of the double entry accounting system and the need to recruit
collaborators. These factors can reduce the unpredictability of the methods used by fraud
perpetrators and can permit the successful application of checklist-based decision aids, as
documented by Eining et al. (1997) and Bell and Carcello (2000). However, we agree that strategic
thinking on the part of the auditor is required.
2
Financial statement fraud is the intentional misstatement or omission of amounts or disclosures in financial
statements, typically by top management, to deceive users. Misappropriation of assets (i.e., employee fraud)
is the second main category of fraud (AICPA 2002). Financial statement fraud is, on average, much more likely
to be material than employee fraud, even though it occurs much less frequently than does employee fraud
(Wells 1997), because top management has the ability to influence larger amounts than do employees, and
the financial misrepresentations designed to mislead users, if successful, are by definition material.
CHECKLIST DESIGN
A checklists design greatly influences its effectiveness (e.g., Glover, Prawitt, and Spilker
1997). The framework in Figure 1 identifies several key checklist design issues that influence the
effectiveness of checklists: customization (related to checklist type), comprehensiveness versus
diagnosticity, and structure of the checklist content.
Customization
Recent discussions in the medical and auditing fields suggest that customizing diagnostic
judgment checklists is necessary to increase their effectiveness (Ely et al. 2011; Cowperthwaite
2012). According to Ely et al. (2011), diagnostic medical checklists are tailored to specific
diseases. In fraud risk assessment and, more generally, in audit settings, customization is
accomplished by adapting the checklist to the characteristics particular to the client and/or the
industry in which the client operates, the staff mix specific to the engagement, and to the areas of
the audit in which professional judgment is important (Cowperthwaite 2012, 39).
3
Cowperthwaite
(2012) suggests that auditors can start with a generic checklist and then tailor it based on factors
such as their knowledge of the client and understanding the requirements of GAAS. The
customization helps to omit irrelevant items in the checklist and emphasize items important to the
engagement (Cowperthwaite 2012).
In the fraud risk assessment task, besides enhancing an auditors risk assessments, an
additional and important benefit of customization is its positive effect on the checklist users ability
to translate a diagnostic judgment into an appropriate audit program. Hammersley (2011) provides
evidence that a red flag fraud checklist can produce appropriate fraud risk assessments, but
should be supplemented with client-specific contextual cues (i.e., the clients circumstance in the
year under audit) in order to effectively translate the fraud risk assessments into appropriate
changes to the audit plan. These benefits of customization are in line with the critique that using
standardized checklists may not reflect an allocation of audit work weighted toward high-risk
areas (PCAOB 2006; emphasis added). In other words, customization can eliminate a mismatch
between the decision aid and the task situation, as well as between the decision aid and the
decision maker, which has been shown to cause passive application of the aid (i.e., its
mechanical use without engaging in active thinking about the task) and inappropriate reliance
(Glover et al. 1997). Some examples of checklists that would be useful are client acceptance/
retention checklists, fraud risk checklists, and internal control evaluation checklists.
Comprehensiveness versus Diagnosticity
Two potentially conflicting design-related issues associated with checklist effectiveness are
comprehensiveness of cues and the relative diagnosticity of positive and negative cues. Pincus
(1989) states that the omission of apparently useful red flags in her checklist highlights the need to
better investigate financial statement fraud indicators, and that developing a comprehensive set of
such indicators might increase the effectiveness of fraud checklists.
4
As the search for useful red
3
The staff mix includes the levels of experience possessed by the audit team members, as well as the
involvement of specialists (e.g., IT specialists, forensic specialists).
4
In Pincus (1989), the participants who did not use the red flag checklist relied on a number of red flags not
included in the checklist, such as public company status, the auditors opinion about certain components of the
clients governance, and the clients cash management skills.
flags continues, auditors should be aware of new types of red flags identified and evaluated by
researchers and regulators, such as disparities between financial and nonfinancial indicators
(Brazel, Jones, and Zimbelman 2009; ASC 2011); e.g., growth in revenue outpacing that in
warehouse space, number of retail outlets, or employee headcount.
However, as desirable as the comprehensiveness of cues in a checklist may be, too many
cues can cause diagnosticity problems because a large proportion of red flags identified in the
literature and in auditing standards are ineffective in distinguishing between fraud and nonfraud
engagements (Bell and Carcello 2000), which raises the issue of red flags predictive ability.
Pincus (1989) notes that her results are consistent with users of checklists under emphasizing
negative indicators suggesting potential fraud. This, in turn, can result in judgment errors at the
cue-combination stage (i.e., the stage at which the auditor synthesizes the cues obtained from all
of the items included in the checklist into a single overall judgment of fraud risk). While there could
be many reasons for such an under emphasis, Pincus (1989) believes that an excessive number of
red flags with weak predictive ability likely draws attention away from those with strong predictive
ability, suggesting that checklist design can be improved by identifying and utilizing a small number
of highly predictive items, and dropping the large number of red flags with limited predictive ability,
as in Bell and Carcello (2000).
5
Pincuss (1989) suggestions are confirmed in more recent studies
by Wood (2012) and Be dard and Graham (2002). Wood (2012) produces evidence that, for a
checklist used in a fraud risk assessment task, nonpredictive cues dilute the predictive value of
diagnostic cues. Be dard and Graham (2002) find that a negatively oriented checklist causes
auditors to identify more risk factors compared to auditors who use a positively oriented checklist, if
the client has high engagement risk.
6
Woods (2012) and Be dard and Grahams (2002) findings suggest that audit firms could
improve their risk assessment procedures through relatively simple modifications to checklist
design. In practice, at least some audit firms appear to be using fraud risk checklists for which red
flag selection is based on statistical procedures that utilize historical data from a variety of their
clients. For example, in Boatsman et al. (1997) an audit firm provided the researchers with an
actual decision aid that used 24 red flag indicators to aid in fraud risk assessment. These indicators
were extracted statistically from a wider set of red flags documented in prior research and
professional literature, resulting in an effective checklist (Boatsman et al. 1997, 219).
7
In addition,
auditors should not disclose details of their audit plans, including their final checklists composition,
to the client. Moreover, if the auditor feels that there is a need to be strategic, then she/he can
collect the information related to red flags with limited predictive ability, but not use them in her/his
final risk assessments (however, this can increase the cost of using checklists).
5
Pincuss (1989) questionnaire is based on the questionnaire in Romney, Albrecht, and Cherrington (1980),
which contains items with a wide variance in predictive ability (Albrecht and Romney 1986). Bell and Carcellos
(2000) checklist relies on a small number of significant red flags (i.e., weak internal control environment, rapid
company growth, inadequate or inconsistent profitability, undue emphasis on meeting earnings projections,
lying to or evasiveness with auditor, aggressive financial reporting, and ownership status).
6
In Be dard and Graham (2002), a checklist has a negative focus if client risk and its consequences are
emphasized and a positive focus if such factors are not emphasized.
7
The aid validation demonstrated a classification accuracy of 81 percent for both fraud and nonfraud audits.
Structure
Boritz (1985) highlights three benefits of applying hierarchical structures to audit checklists:
structures clarify problems through decomposition and modularization of information sets;
structures limit the use of heuristics and biases of auditors; structures incorporate knowledge
and expertise that highlight relationships and information cues. Wilks and Zimbelman (2004)
provide evidence that structural improvements can make fraud risk assessment checklists more
effective. In their experiment, audit managers who used fraud checklists in conjunction with the
decomposition of fraud risk assessments according to the fraud triangle (which postulates that
three conditions must be present for a fraud to occur: incentive, opportunity, and attitude)
assessed fraud risk better than did managers who relied on checklists without such decomposition
(holistic approach), but only when opportunity and incentive cues indicated low risk. This result
demonstrates that the decomposition aid is more effective than a traditional checklist in certain
contexts.
8
In extant auditing research, this study often is cited as evidence that checklists are not
useful (Hogan et al. 2008). However, we believe that Wilks and Zimbelmans (2004) results
indicate that improvements in cue organization can increase a checklists effectiveness.
CHECKLIST APPLICATION
The framework in Figure 1 identifies two key checklist application issues that influence the
effectiveness of checklists: the method of combining cues and individual versus group use of
checklists.
Method of Combining Cues
Kahneman (2011) describes two reasoning systems that govern cue processing: an intuitive
system (System 1) that rapidly combines cues into a composite based on prior knowledge,
heuristics, stereotypes, expectancies, scripts, and schemas (Bargh and Chartrand 1999), and a
deliberative system (System 2) that combines cues in accordance with a rational decision model.
Because of its speed and efficiency, the intuitive reasoning system often preempts the deliberative
system and leads people to rely on heuristics and biases. Sloman (1996) points out that, even
when a person tries to be rule governed, responses prompted by unconscious associations with
information cues encroach on judgment quality. This conclusion leads to the rationale for using
decision aids to compensate for the risks associated with the application of intuitive judgment to
combine the cues identified with the help of checklists.
Recent behavioral studies suggest that deliberative thinking can be induced to improve fraud
risk assessments. For example, if opportunistic managers anticipate and take advantage of a
traditional risk-based audit strategy by concealing misstatements within low-risk accounts more
often than within high-risk accounts, auditors can successfully predict and counter such managers
expectations if they are prompted to think strategically about managers expected responses to
their audit approach (Hoffman and Zimbelman 2009, 2012). This supposition suggests that
checklists usefulness for fraud risk assessments can be increased if the checklists are designed to
encourage deliberative, strategic reasoning about the appropriate audit response to identified
risks. One possible way to accomplish this is to incorporate items that help auditors identify and
8
The decomposition is distinct from grouping checklist items, which has been incorporated in the standards
(AICPA 2002; Wilks and Zimbelman 2004, 741).
direct their attention to accounts and classes of transactions for which the traditional audit
approach is expected to produce a low-risk estimate.
Beyond inducing deliberative instead of intuitive cue weighting and combination, is the
possibility of using computational models. Extant psychology research suggests that individuals
are not skilled at identifying complex, nonlinear relationships within data (Hammond and Summers
1972). Therefore, simple, additive models utilizing limited information, such as those based on
linear regression, frequently perform better at identifying relationships than do experts, even when
a large amount of data is available (e.g., Dawes 1971). Pincus (1989) mentions that she did not
have a reliable empirical cue-processing model at her disposal when she conducted her study;
therefore, cues were evaluated using only the auditors judgment. In contrast, Eining et al. (1997)
and Bell and Carcello (2000) had two different but reliable cue-processing models and were able to
compare the usefulness of three types of fraud cue-processing aids: a checklist, a statistical
model, and an expert system. Eining et al. (1997) and Bell and Carcello (2000) confirm that auditor
judgment based on using the checklist alone is inferior to unaided judgment; however, the fraud
risk assessments obtained with the statistical model and the expert system are far superior to
those obtained through the use of unaided judgment.
9
The key point that must be emphasized
here is that the checklist-based identification of cues is necessary (although not sufficient) for the
effective functioning of both the statistical model and the expert system. In other words, the
problem with the red flag checklist is not the use of a checklist per se but the weak design of the
checklist and the ineffective combination of the cues identified using the checklist.
Using cue-combination models also can benefit other types of audit checklists including those
intended for client acceptance and continuation judgments, control risk evaluation, and planning of
substantive tests. For client acceptance and continuation, Bell et al. (2002) discuss a sophisticated
decision aid developed by KPMG that decomposes the decision into several categories and
subcategories. This process involves an extensive set of checklists that include information
gathered by the audit staff, which is then combined into an overall judgment of acceptance/
continuation risk by a set of mathematical algorithms. For planning checklists, Bonner et al. (1996)
find that a checklist combined with a model for aggregating judgment components performs
significantly better than unaided judgment, while a checklist alone gives only a slight improvement,
because of the ability of the combined aid to facilitate both knowledge retrieval and aggregation.
For control checklists, Ashton (1974) states that a likely reason for inconsistencies in internal
control risk judgments among auditors is differential weightings of the internal control indicators
they use. He suggests that a possible solution is to rely on a model that approximates true
statistical weights for the combination of indicators into an overall judgment (Ashton 1974, 153).
Libby and Libby (1989) demonstrate that making component judgments of the strength of the
individual controls, and then aggregating the component judgments using a mechanical model is
superior to audit seniors global control risk assessments.
These results provide two key insights into the usefulness of audit checklists. First, it appears
that auditors judgment processes, when using checklist-gathered evidence, breaks down when
they combine the cues obtained from the checklist into a final judgment. Second, the cue-
combination models and expert systems are, in essence, a combination of two decision aidsa
checklist for finding the cues (knowledge retrieval) and a model for aggregating the set of cues
9
Interestingly, a contemporaneous study by Shelton, Whittington, and Landsittel (2001) demonstrates that, in
practice, Big N and second-tier audit firms did not rely widely on expert systems and other tools for processing
cues obtained from their red flag checklists.
(knowledge aggregation). For the model to be effective, an appropriately small set of diagnostic
cues to be processed by the model is required. Thus, a suitably designed audit checklist applied
jointly with an effective cue-combination model can be a useful tool in conducting audits. In
addition, it can reduce liability, as will be discussed later.
Individual versus Group Judgment
In the medical field, both procedural operating room and judgmental diagnostic checklists are
used in group settings. Such settings involve doctors specializing in different clinical disciplines,
nurses, and patients, who tend to use distinct terminologies to describe the same phenomena,
thus causing miscommunication because of translation errors (Winters et al. 2009). Winters et al.
(2009) suggest that checklists help to democratize knowledge by standardizing and facilitating
the reliable translation of information, so that the same knowledge awareness of the importance of
a given set of cues is imparted to persons with different perspectives, professional cultures, and
educational backgrounds. There also are suggestions that general diagnostic checklists that
prompt doctors to optimize their cognitive approach can reduce groupthink, whereby a diagnosis
by one member of a group of doctors is hastily adopted by the others (Croskerry 2007; Ely et al.
2011).
In auditing, as in medicine, many tasks involve multiperson teams. Prior research has shown
that mathematical composites of checklist-based individual judgments can outperform individuals
in judgment tasks, such as internal control evaluation (Trotman et al. 1983). Fraud risk
assessments in financial audits are, at least in part, an interactive group process, as SAS No. 99
(AICPA 2002) stipulates that auditors must discuss the potential for material misstatement caused
by fraudulent activities, including an exchange of ideas or brainstorming among the members of
the audit team. These brainstorming sessions have been shown to be effective (Hoffman and
Zimbelman 2009, 2012), and a positive role of checklists in such group judgment and decision
making is their ability to reduce miscommunication. Alon and Dwyer (2010) investigate how the
brainstorming component of SAS No. 99 (AICPA 2002) influences checklist use and reliance, and
the effectiveness of fraud risk assessments. Using an approach similar to that employed by Pincus
(1989), Alon and Dwyer (2010) find that two-person groups of students with a checklist decision aid
outperform individuals with and without the decision aid, because of information processing gains
arising from the group interaction. Further, groups with the decision aid identified more quality
fraud ideas than those without the aid, suggesting that checklists add to decision quality for
brainstorming groups (Alon and Dwyer 2010, 248). These results counter Bamber et al.s (2007)
argument that the SAS No. 99 brainstorming requirement reduces the importance of fraud
checklists.
IMPACT OF CONTEXTUAL FACTORS
Psychology research recognizes the importance of contextual factors for both individual and
group judgment and decision making (Kerr and Tindale 2004; Bonner 2008), implying that these
factors also influence the use of checklists. The framework in Figure 1 identifies the following three
key contextual factors that affect checklist use in auditing: performance pressures, legal liability
considerations, and auditor characteristics (e.g., Ng and Tan 2003; Hammersley 2011).
10
10
Performance pressures are those arising from accountability structures, time pressures, and financial and
other incentives.
Performance Pressures
Gawande (2009) reports that procedural checklists are very effective in high-pressure work
environments (e.g., operating rooms, airplane cockpits). These checklists improve outcomes
because they draw users attention to tasks that have to be completed, but may often be
overlooked if there is no checklist. This creates an expectation that such checklists would perform
well in other high-pressure workplaces, such as those encountered by auditors. Surprisingly,
research studying how procedural checklists are and should be used in audits is almost
nonexistent. McDaniel (1990) investigates how the joint imposition of program structure (consisting
of detailed procedures and/or checklists) and time constraints affects auditors performance.
McDaniel finds that, when time pressure is low, structure significantly increases audit
effectiveness, efficiency, and consistency, and when time pressure increases, audit efficiency
increases and effectiveness decreases. Structure is associated with smaller decreases in audit
effectiveness at higher levels of time pressure, but the difference is not significant (McDaniel 1990,
268). This result is somewhat inconsistent with the effective performance of procedural checklists
in the high-pressure tasks described in Gawande (2009), so the issue requires further
investigation. Blocher et al. (1983) find that when auditors use procedural checklists for planning
analytical review, they plan more analytical and more tests of details procedures, compared to
when they do not use such checklists.
Legal Liability Considerations
Sutton, Young, and McKenzie (1994) investigate liability issues arising from an audit firms
use of an expert system (as discussed, such systems may be based on structured checklists) and
identify two sources of risk. One is that the joint judgment of the auditor and the expert system may
not supply the required level of expertise, and the auditor will be held liable under GAAS fieldwork
standards (AICPA 1993; PCAOB 2001) for not properly planning and supervising the use of the
system. The other is that not relying on an expert system (when one is available) may be turned
against the auditor because the existence of such a system can increase the level of expertise a
prudent practitioner should exercise. Jennings, Kneer, and Reckers (1993) find that, when GAAS
standards are not readily at hand for a particular area, jurists use audit firms internal decision aids
as surrogate standards against which to gauge auditors performance. Results in Lowe, Reckers,
and Whitecotton (2002) indicate that, if a decision aid is reliable, and the auditor followed its
recommendations, the jurors attributed lower responsibility to the auditor. This indicates that jurors
place a high value on decision aids and expect auditors to use them when effective ones are
available.
Auditor Characteristics
Some auditors are very hesitant to rely on the judgment output of mechanical decision aids,
particularly when faced with performance pressures such as financial incentives, justification, and
outcome feedback.
11
More experienced auditors appear to rely on decision aids less because of
their confidence, perhaps even overconfidence, in their own expertise. Boatsman et al. (1997)
found that auditors were more convinced of the credibility of the checklist-based decision aid when
11
Nelson and Tan (2005) provide a literature review that includes a section on decision aids; however, a recent
review by Knechel, Krishnan, Pevzner, Shefchik, and Velury (2013) barely mentions them.
it predicted no fraud rather than fraud and their reliance on a decision aid was a function of their
concern about the severity of anticipated penalties for incorrect decisions.
12
From a practical point
of view, an interesting finding of the study is that a simultaneous increase in the penalties for audit
failure and for over auditing generates a higher reliance on the aid. Kaplan, Reneau, and
Whitecotton (2001) find that auditors with an external locus of control rely more on a mechanical
cue-combination decision aid than those with an internal locus of control, and that involving
decision makers in the aids development enhances reliance (the effect is more pronounced for
internal locus of control auditors). Sieck and Arkes (2005) discover that peoples overconfidence in
the quality of their intuitive judgment adds to their reluctance to use effective cue-combining aids,
and that only enhanced calibration feedback (involving answering several questions from memory)
reduced overconfidence and increased reliance on the aid. Gomaa, Hunton, and Rose (2008) find
that practitioners rely more on decision aids when either litigation risk or control risk is high; when
both risks are high, litigation risk amplifies the auditors awareness of legal defensibility and, thus,
increases decision aid reliance, even though the auditors confidence in the quality of their
judgment deteriorates. In other words, there is a tension between the overconfidence that could
lead auditors to ignore or override decision aids and the pressure to reduce legal liability that could
lead auditors to subordinate their judgment to that of the checklist.
RECOMMENDATIONS AND CONCLUSION
Our review of existing audit research indicates that procedural checklists are generally
thought to perform satisfactorily; whereas, there is a widely held view that the use of diagnostic
checklists for fraud risk assessment in financial audits yields dysfunctional outcomes (e.g., Hogan
et al. 2008; Jamal 2008), and that this view is spreading to other uses of checklists (e.g., Wheeler
and Arunachalam 2008; Seow 2011). However, we believe that abandoning the use of checklists,
such as red flag checklists, in favor of unaided judgment may not be the best option for large firms
with multilocation audits that require a degree of standardization and coordination. Our analysis of
the literature along dimensions of the suggested framework in Figure 1 identified various strategies
to improve checklist effectiveness. These strategies are presented in Table 2, using the fraud risk
assessment checklist as an illustrative example. Below, we highlight some of the key
recommendations in this table.
Our analysis indicates that cue processing appears to be the weak link in the use of
judgmental checklists, implying that they need to be integrated with a cue-combining model, an
algorithm based on a statistical model or expert system, to achieve better performance. If an
appropriate cue-combining model is not available, then checklists need to be designed to trigger
deliberative strategic thinking on the part of the auditor to better counter the strategic nature of
management behavior. For example, in a fraud risk assessment context, this could be
accomplished by encouraging auditors to critically address accounts that the traditional audit
approach labels as low risk, or to consider managements anticipation of their audit program
changes in response to identified risks.
Another option is to use groups to mitigate the weaknesses associated with unaided individual
judgment (such as in a fraud risk brainstorming session), even though groups are themselves
subject to a number of factors that can reduce judgmental group performance (Kerr and Tindale
2004). Checklists can exert positive effects on group judgment by mitigating miscommunication,
12
Boatsman et al.s (1997) decision aid consists of a list of 24 red flags with a subsequent mechanical
combination of cues to estimate fraud potential. It was provided to the authors by an audit firm.
T
A
B
L
E
2
O
b
j
e
c
t
i
v
e
s
o
f
a
n
d
S
t
r
a
t
e
g
i
e
s
f
o
r
I
m
p
r
o
v
i
n
g
E
f
f
e
c
t
i
v
e
n
e
s
s
o
f
D
i
a
g
n
o
s
t
i
c
C
h
e
c
k
l
i
s
t
s
,
U
s
i
n
g
a
R
e
d
F
l
a
g
C
h
e
c
k
l
i
s
t
a
s
a
n
I
l
l
u
s
t
r
a
t
i
v
e
E
x
a
m
p
l
e
F
a
c
t
o
r
s
A
f
f
e
c
t
i
n
g
C
h
e
c
k
l
i
s
t
U
s
e
F
i
n
d
i
n
g
s
i
n
P
r
i
o
r
L
i
t
e
r
a
t
u
r
e
I
m
p
r
o
v
e
m
e
n
t
O
b
j
e
c
t
i
v
e
s
S
t
r
a
t
e
g
i
e
s
f
o
r
I
m
p
r
o
v
e
m
e
n
t
C
h
a
r
a
c
t
e
r
i
s
t
i
c
s
o
f
T
a
s
k
(
e
.
g
.
,
F
r
a
u
d
R
i
s
k
A
s
s
e
s
s
m
e
n
t
)
E
c
o
n
o
m
i
c
g
a
m
e
t
h
e
o
r
y
m
o
d
e
l
s
f
r
a
u
d
c
r
e
a
t
i
o
n
a
s
a
s
t
r
a
t
e
g
i
c
a
c
t
i
v
i
t
y
(
J
a
m
a
l
2
0
0
8
)
,
b
u
t
a
u
d
i
t
o
r
s
c
a
n
s
u
c
c
e
s
s
f
u
l
l
y
c
o
u
n
t
e
r
m
a
n
a
g
e
r
s
a
c
t
i
o
n
s
w
h
e
n
p
r
o
m
p
t
e
d
t
o
t
h
i
n
k
s
t
r
a
t
e
g
i
c
a
l
l
y
a
b
o
u
t
m
a
n
a
g
e
r
s
r
e
s
p
o
n
s
e
s
t
o
t
h
e
a
u
d
i
t
a
p
p
r
o
a
c
h
(
H
o
f
f
m
a
n
a
n
d
Z
i
m
b
e
l
m
a
n
2
0
0
9
,
2
0
1
2
)
.
I
n
c
r
e
a
s
e
c
h
e
c
k
l
i
s
t
e
f
f
e
c
t
i
v
e
n
e
s
s
a
t
c
o
u
n
t
e
r
i
n
g
m
a
n
a
g
e
m
e
n
t
s
t
r
a
t
e
g
i
c
f
r
a
u
d
-
c
r
e
a
t
i
n
g
b
e
h
a
v
i
o
r
.
D
e
s
i
g
n
c
h
e
c
k
l
i
s
t
s
t
o
e
n
c
o
u
r
a
g
e
s
t
r
a
t
e
g
i
c
r
e
a
s
o
n
i
n
g
,
f
o
r
e
x
a
m
p
l
e
b
y
i
n
c
o
r
p
o
r
a
t
i
n
g
i
t
e
m
s
f
o
c
u
s
i
n
g
t
h
e
a
u
d
i
t
o
r
s
a
t
t
e
n
t
i
o
n
o
n
a
c
c
o
u
n
t
s
a
n
d
c
l
a
s
s
e
s
o
f
t
r
a
n
s
a
c
t
i
o
n
s
t
h
a
t
a
r
e
l
o
w
r
i
s
k
u
n
d
e
r
t
h
e
t
r
a
d
i
t
i
o
n
a
l
a
u
d
i
t
a
p
p
r
o
a
c
h
.
C
h
e
c
k
l
i
s
t
D
e
s
i
g
n
:
T
y
p
e
D
i
a
g
n
o
s
t
i
c
c
h
e
c
k
l
i
s
t
s
,
s
u
c
h
a
s
f
r
a
u
d
c
h
e
c
k
l
i
s
t
s
,
m
a
y
r
e
q
u
i
r
e
c
u
s
t
o
m
i
z
a
t
i
o
n
i
n
o
r
d
e
r
t
o
b
e
e
f
f
e
c
t
i
v
e
(
E
l
y
e
t
a
l
.
2
0
1
1
;
C
o
w
p
e
r
t
h
w
a
i
t
e
2
0
1
2
)
.
I
m
p
r
o
v
e
t
h
e
t
b
e
t
w
e
e
n
t
h
e
f
r
a
u
d
c
h
e
c
k
l
i
s
t
a
n
d
t
h
e
f
r
a
u
d
r
i
s
k
a
s
s
e
s
s
m
e
n
t
t
a
s
k
,
a
s
w
e
l
l
a
s
b
e
t
w
e
e
n
t
h
e
f
r
a
u
d
c
h
e
c
k
l
i
s
t
a
n
d
t
h
e
u
s
e
r
-
a
u
d
i
t
o
r
(
G
l
o
v
e
r
e
t
a
l
.
1
9
9
7
)
.
C
u
s
t
o
m
i
z
e
t
h
e
c
h
e
c
k
l
i
s
t
t
o
t
h
e
c
l
i
e
n
t
,
c
l
i
e
n
t
s
i
n
d
u
s
t
r
y
,
a
n
d
e
n
g
a
g
e
m
e
n
t
/
a
u
d
i
t
r
m
s
t
a
f
f
m
i
x
(
C
o
w
p
e
r
t
h
w
a
i
t
e
2
0
1
2
)
.
C
h
e
c
k
l
i
s
t
D
e
s
i
g
n
:
C
o
n
t
e
n
t
/
S
t
r
u
c
t
u
r
e
T
r
a
d
i
t
i
o
n
a
l
f
r
a
u
d
c
h
e
c
k
l
i
s
t
s
m
a
y
o
m
i
t
a
n
u
m
b
e
r
o
f
h
i
g
h
p
r
e
d
i
c
t
i
v
e
a
b
i
l
i
t
y
r
e
d
a
g
s
(
P
i
n
c
u
s
1
9
8
9
)
;
h
o
w
e
v
e
r
,
a
n
e
x
c
e
s
s
i
v
e
n
u
m
b
e
r
o
f
n
o
n
d
i
a
g
n
o
s
t
i
c
r
e
d
a
g
s
m
a
y
s
u
p
p
r
e
s
s
t
h
e
d
i
a
g
n
o
s
t
i
c
i
t
y
o
f
d
i
a
g
n
o
s
t
i
c
r
e
d
a
g
s
.
C
a
t
e
g
o
r
i
z
i
n
g
c
u
e
s
i
n
t
o
f
r
a
u
d
t
r
i
a
n
g
l
e
c
a
t
e
g
o
r
i
e
s
c
a
n
i
m
p
r
o
v
e
j
u
d
g
m
e
n
t
s
(
W
i
l
k
s
a
n
d
Z
i
m
b
e
l
m
a
n
2
0
0
4
)
C
r
e
a
t
e
a
m
o
r
e
c
o
m
p
r
e
h
e
n
s
i
v
e
l
i
s
t
o
f
h
i
g
h
p
r
e
d
i
c
t
i
v
e
a
b
i
l
i
t
y
r
e
d
a
g
s
,
b
u
t
a
v
o
i
d
n
o
n
p
r
e
d
i
c
t
i
v
e
r
e
d
a
g
s
.
I
n
c
l
u
d
e
a
n
d
t
e
s
t
n
e
w
l
y
d
i
s
c
o
v
e
r
e
d
r
e
d
a
g
s
s
u
c
h
a
s
t
h
e
d
i
s
p
a
r
i
t
y
i
n
g
r
o
w
t
h
b
e
t
w
e
e
n
n
a
n
c
i
a
l
a
n
d
n
o
n
n
a
n
c
i
a
l
i
n
d
i
c
a
t
o
r
s
(
B
r
a
z
e
l
e
t
a
l
.
2
0
0
9
;
A
S
C
2
0
1
1
)
;
e
l
i
m
i
n
a
t
e
n
o
n
p
r
e
d
i
c
t
i
v
e
r
e
d
a
g
s
.
S
t
r
u
c
t
u
r
e
c
h
e
c
k
l
i
s
t
t
o
e
n
h
a
n
c
e
c
u
e
d
i
a
g
n
o
s
t
i
c
i
t
y
.
S
t
r
u
c
t
u
r
e
/
c
a
t
e
g
o
r
i
z
e
c
u
e
s
u
s
i
n
g
a
p
p
r
o
p
r
i
a
t
e
c
a
t
e
g
o
r
i
e
s
.
(
c
o
n
t
i
n
u
e
d
o
n
n
e
x
t
p
a
g
e
)
T
A
B
L
E
2
(
c
o
n
t
i
n
u
e
d
)
F
a
c
t
o
r
s
A
f
f
e
c
t
i
n
g
C
h
e
c
k
l
i
s
t
U
s
e
F
i
n
d
i
n
g
s
i
n
P
r
i
o
r
L
i
t
e
r
a
t
u
r
e
I
m
p
r
o
v
e
m
e
n
t
O
b
j
e
c
t
i
v
e
s
S
t
r
a
t
e
g
i
e
s
f
o
r
I
m
p
r
o
v
e
m
e
n
t
A
p
p
l
i
c
a
t
i
o
n
:
M
e
t
h
o
d
o
f
C
o
m
b
i
n
i
n
g
C
u
e
s
A
u
d
i
t
o
r
s
r
e
l
y
i
n
g
o
n
f
r
a
u
d
c
h
e
c
k
l
i
s
t
s
i
n
c
o
m
b
i
n
a
t
i
o
n
w
i
t
h
s
t
a
t
i
s
t
i
c
a
l
m
o
d
e
l
s
a
n
d
e
x
p
e
r
t
s
y
s
t
e
m
s
f
o
r
c
o
m
b
i
n
i
n
g
t
h
e
c
u
e
s
r
e
s
u
l
t
i
n
a
s
u
p
e
r
i
o
r
r
i
s
k
a
s
s
e
s
s
m
e
n
t
c
o
m
p
a
r
e
d
t
o
a
u
d
i
t
o
r
s
c
o
m
b
i
n
i
n
g
t
h
e
c
u
e
s
i
n
t
u
i
t
i
v
e
l
y
(
E
i
n
i
n
g
e
t
a
l
.
1
9
9
7
;
B
e
l
l
a
n
d
C
a
r
c
e
l
l
o
2
0
0
0
)
.
R
e
d
u
c
e
j
u
d
g
m
e
n
t
e
r
r
o
r
s
i
n
c
u
e
-
c
o
m
b
i
n
a
t
i
o
n
p
r
o
c
e
s
s
a
r
i
s
i
n
g
f
r
o
m
d
i
f
c
u
l
t
i
e
s
w
i
t
h
i
d
e
n
t
i
f
y
i
n
g
c
o
m
p
l
e
x
,
n
o
n
l
i
n
e
a
r
r
e
l
a
t
i
o
n
s
h
i
p
s
i
n
t
h
e
d
a
t
a
(
H
a
m
m
o
n
d
a
n
d
S
u
m
m
e
r
s
1
9
7
2
)
.
D
e
s
i
g
n
c
h
e
c
k
l
i
s
t
s
t
o
e
n
c
o
u
r
a
g
e
d
e
l
i
b
e
r
a
t
i
v
e
r
a
t
h
e
r
t
h
a
n
i
n
t
u
i
t
i
v
e
c
u
e
p
r
o
c
e
s
s
i
n
g
.
D
e
v
e
l
o
p
a
n
d
i
m
p
l
e
m
e
n
t
s
t
a
t
i
s
t
i
c
a
l
m
o
d
e
l
s
a
n
d
e
x
p
e
r
t
s
y
s
t
e
m
s
t
o
a
s
s
i
s
t
i
n
d
i
v
i
d
u
a
l
a
u
d
i
t
o
r
s
w
i
t
h
c
u
e
c
o
m
b
i
n
a
t
i
o
n
.
A
p
p
l
i
c
a
t
i
o
n
:
I
n
d
i
v
i
d
u
a
l
v
e
r
s
u
s
T
e
a
m
C
h
e
c
k
l
i
s
t
s
h
a
v
e
a
p
o
t
e
n
t
i
a
l
t
o
i
m
p
r
o
v
e
g
r
o
u
p
j
u
d
g
m
e
n
t
t
h
r
o
u
g
h
d
e
m
o
c
r
a
t
i
z
i
n
g
k
n
o
w
l
e
d
g
e
a
n
d
e
l
i
m
i
n
a
t
i
n
g
g
r
o
u
p
t
h
i
n
k
(
E
l
y
e
t
a
l
.
2
0
1
1
;
C
r
o
s
k
e
r
r
y
2
0
0
7
)
,
w
h
i
l
e
f
r
a
u
d
b
r
a
i
n
s
t
o
r
m
i
n
g
s
e
s
s
i
o
n
s
m
a
y
s
u
f
f
e
r
f
r
o
m
m
i
s
c
o
m
m
u
n
i
c
a
t
i
o
n
b
e
t
w
e
e
n
a
u
d
i
t
o
r
s
a
n
d
f
o
r
e
n
s
i
c
s
p
e
c
i
a
l
i
s
t
s
(
B
o
r
i
t
z
,
K
o
c
h
e
t
o
v
a
-
K
o
z
l
o
s
k
i
,
a
n
d
R
o
b
i
n
s
o
n
2
0
1
3
)
.
I
m
p
r
o
v
e
g
r
o
u
p
p
e
r
f
o
r
m
a
n
c
e
v
i
a
b
e
t
t
e
r
i
n
t
r
a
g
r
o
u
p
c
o
m
m
u
n
i
c
a
t
i
o
n
a
n
d
r
e
d
u
c
e
d
u
n
d
e
s
i
r
a
b
l
e
e
f
f
e
c
t
s
s
u
c
h
a
s
g
r
o
u
p
t
h
i
n
k
(
K
e
r
r
a
n
d
T
i
n
d
a
l
e
2
0
0
4
)
.
D
e
s
i
g
n
r
e
d
a
g
c
h
e
c
k
l
i
s
t
s
t
o
f
a
c
i
l
i
t
a
t
e
c
o
m
m
u
n
i
c
a
t
i
o
n
b
e
t
w
e
e
n
a
u
d
i
t
o
r
s
a
n
d
f
o
r
e
n
s
i
c
s
p
e
c
i
a
l
i
s
t
s
a
t
f
r
a
u
d
b
r
a
i
n
s
t
o
r
m
i
n
g
s
e
s
s
i
o
n
s
a
n
d
b
e
t
w
e
e
n
o
t
h
e
r
t
e
a
m
m
e
m
b
e
r
s
.
C
o
n
t
e
x
t
u
a
l
F
a
c
t
o
r
s
:
A
u
d
i
t
o
r
s
E
x
p
e
r
i
e
n
c
e
/
E
x
p
e
r
t
i
s
e
;
P
e
r
f
o
r
m
a
n
c
e
P
r
e
s
s
u
r
e
;
L
e
g
a
l
C
o
n
s
i
d
e
r
a
t
i
o
n
s
C
o
n
t
e
x
t
u
a
l
f
a
c
t
o
r
s
,
s
u
c
h
a
s
t
h
e
a
u
d
i
t
o
r
s
e
x
p
e
r
i
e
n
c
e
,
p
e
r
f
o
r
m
a
n
c
e
p
r
e
s
s
u
r
e
s
,
a
n
d
l
e
g
a
l
c
o
n
s
i
d
e
r
a
t
i
o
n
s
,
i
m
p
a
c
t
t
h
e
d
e
g
r
e
e
o
f
r
e
l
i
a
n
c
e
p
l
a
c
e
d
o
n
f
r
a
u
d
c
h
e
c
k
l
i
s
t
s
b
y
a
u
d
i
t
o
r
s
(
B
o
a
t
s
m
a
n
e
t
a
l
.
1
9
9
7
;
L
o
w
e
e
t
a
l
.
2
0
0
2
)
.
R
e
d
u
c
e
i
n
e
f
c
i
e
n
t
u
s
e
o
f
f
r
a
u
d
c
h
e
c
k
l
i
s
t
s
w
h
e
r
e
b
y
t
h
e
f
r
a
u
d
r
i
s
k
a
s
s
e
s
s
m
e
n
t
i
s
o
b
t
a
i
n
e
d
b
u
t
n
o
t
r
e
l
i
e
d
u
p
o
n
.
E
x
p
l
i
c
i
t
l
y
c
o
n
s
i
d
e
r
c
o
n
t
e
x
t
u
a
l
f
a
c
t
o
r
s
w
h
e
n
a
p
p
l
y
i
n
g
f
r
a
u
d
c
h
e
c
k
l
i
s
t
s
.
groupthink, or other factors known to reduce judgmental group performance. In particular, they
may mitigate miscommunication between various specialists (e.g., forensic specialists, tax
specialists, valuation specialists) and regular audit team members during the risk assessment
planning phase of the audit.
Contextual factors, such as performance pressures and the legal environment, need to be
considered by audit team managers, as they can affect the degree of reliance placed by audit team
members on checklists and impact the translation of assessed risk into changes in the audit
program. In audit settings, the use of checklists renders an additional benefit of fulfilling
documentation requirements imposed by regulatory and standard-setting bodies, such as the
PCAOB. The application of judgmental checklists can produce necessary evidence supporting
auditors conclusions in the areas in which appropriate documentation is currently lacking. For
example, red flag checklists can be useful for documenting the estimate of risk of material
misstatement because of fraud, while checklists designed for fair value auditing (e.g., PwC 2011;
Deloitte 2012) can improve the documentation in that area.
Many important questions related to the use of audit checklists remain unexplored and open
for future research. Cowperthwaite (2012) asserts that customization may mitigate some of the
weaknesses of generic red flag checklists while retaining some of their benefits; however, such
customization has not been researched in audit settings and raises several questions.
13
If the
engagement team is qualified to modify the checklist (and is allowed by their firm to do so), then
why would they not be able to make good use of the standard checklist? Will audit teams take the
time needed to modify checklists appropriately? If the engagement team modifies the checklist,
might they make it worse, not better? These questions suggest that much useful research could be
conducted in this area. Gawande (2009, 120) asserts that a good checklist must possess two
qualitiesit should be short and precise. However, among the many studies discussed in this
commentary, only Bell and Carcello (2000) and Boatsman et al. (1997) address the issue to some
extent, reflecting on the theoretical and practical aspects of cues diagnosticity. Clear criteria for
the precision and inclusion/exclusion of questions in an audit checklist are yet to be developed.
Another area for research is the communication-enhancing role of checklistsboth the
communication between different types of specialists and between supervisors and subordinates
(Gawande 2009, 67, 103; KPMG 2011). Evidence needs to be obtained on what role checklists
can play in integrating the diverse personnel involved in the audit process.
REFERENCES
Albrecht, W. S., and M. B. Romney. 1986. Red-flagging management fraud: A validation. Advances in Accounting 3:
323333.
Alon, A., and P. Dwyer. 2010. The impact of groups and decision aid reliance on fraud risk assessment. Management
Research Review 33 (3): 240256.
American Institute of Certified Public Accountants (AICPA). 1993. Codification of Statements on Auditing Standards.
New York, NY: AICPA.
American Institute of Certified Public Accountants (AICPA). 2002. Consideration of Fraud in a Financial Statement
Audit. Statement on Auditing Standards No. 99. New York, NY: AICPA.
Anderson, R. J. 1977. The External Audit 1: Concepts and Techniques. Toronto, ON: Pitman.
Asare, S., and A. Wright. 2004. The effectiveness of alternative risk assessment and program planning tools in a fraud
setting. Contemporary Accounting Research 21 (2): 325352.
13
We are grateful to the editor for suggesting these questions.
Ashton, R. H. 1974. An Experimental study of internal control judgments. Journal of Accounting Research 12 (1): 143
157.
Auditing Standards Committee (ASC). 2011. RE: Concept Release on Possible Revisions to PCAOB Standards
Related to Reports on Audited Financial. PCAOB Rulemaking Docket Matter No. 34; PCAOB Release No. 2011-
003. Available at: http://pcaobus.org/Rules/Rulemaking/Docket034/Concept_Release.pdf
Bamber, E. M., T. D. Carpenter, and J. S. Hammersley. 2007. The Influence of Documentation Specificity and Fraud
Risk Priming on Auditor Fraud Judgments and Evidence Evaluation Decisions. Working paper, J. M. Tull School
of Accounting and The University of Georgia Terry College of Business.
Bargh, J. A., and T. L. Chartrand. 1999. The unbearable automaticity of being. American Psychologist 54 (7): 462
479.
Be dard, J. C., and L. E. Graham. 2002. The effects of decision aid orientation on risk factor identification and audit test
planning. Auditing: A Journal of Practice & Theory 21 (2): 3956.
Bell, T. B., and J. V. Carcello. 2000. A decision aid for assessing the likelihood of fraudulent financial reporting.
Auditing: A Journal of Practice & Theory 19 (1): 169184.
Bell, T. B., J. C. Be dard, K. M. Johnstone, and E. F. Smith. 2002. KRisk(SM): A computerized decision aid for client
acceptance and continuance. Auditing: A Journal of Practice & Theory 21 (2): 97113.
Blocher, E., R. S. Esposito, and J. J. Willingham. 1983. Auditors analytical review judgments for payroll expense.
Auditing: A Journal of Practice & Theory 3 (1): 7591.
Boatsman, J., C. Moeckel, and B. Pei. 1997. The effect of decision consequences on auditors reliance on decision
aids in audit planning. Organizational Behavior and Human Decision Processes 71 (2).
Bonner, S. E. 2008. Judgment and Decision Making in Accounting. Upper Saddle River, NJ: Pearson Education, Inc.
Bonner, S. E., R. Libby, and M. W. Nelson. 1996. Using decision aids to improve auditors conditional probability
judgments. The Accounting Review 71 (2): 221240.
Boritz, J. E. 1985. The effect of information presentation structures on audit planning and review judgments.
Contemporary Accounting Research l (2): 193218.
Boritz, J. E., A. Albuquerque, and R. G. Kielstra. 1991. A prototype expert system for the assessment of inherent risk
and prior probability of error. In Decision Support and Expert Systems for Management Accountants, edited by
Armitage, H. M., and J. E. Boritz. Toronto, ON: Society of Management Accountants of Canada.
Boritz, J. E., N. Kochetova-Kozloski, and L. A. Robinson. 2013. Are Fraud Specialists Relatively More Effective at
Modifying Audit Programs in the Presence of Fraud Risk? Working paper, University of Waterloo School of
Accounting and Finance and Saint Marys University Sobey School of Business.
Brazel, J. F., K. L. Jones, and M. F. Zimbelman. 2009. Using nonfinancial measures to assess fraud risk. Journal of
Accounting Research 47: 11351166.
Butler, S. A. 1985. Application of a decision aid in the judgmental evaluation of substantive test of details samples.
Journal of Accounting Research 23 (Autumn): 513526.
Carvalho, P. V. R., I. L. dos Santos, and M. C.R. Vidal. 2005. Nuclear power plant shift supervisors decision making
during microincidents. International Journal of Industrial Ergonomics 35: 619644.
Carvalho, P. V. R., M. C.R. Vidal, and E. F. de Carvalho. 2007. Nuclear power plant communications in normative and
actual practice: A field study of control room operators communications. Human Factors and Ergonomics in
Manufacturing 17 (1): 4378.
Cowperthwaite, P. 2012. Are checklists killing our profession? CA Magazine (September): 3839.
Croskerry, P. 2007. Timely recognition and diagnosis of illness. In Safe and Effective: The Eight Essential Elements of
an Optimal Medication-Use System, edited by N. MacKinnon, 7993. Ottawa, ON: Canadian Pharmacists
Association.
Dawes, R. 1971. A case study of graduate admissions: Application of three principles of decision making. American
Psychologist 26: 180188.
Deloitte LLP (Deloitte). 2012. Qualitative Goodwill Impairment Assessment. Available at: http://www.iasplus.com/en/
publications/us/roadmap-series/qualitative-goodwill-impairment-assessment
Eining, M. M., D. R. Jones, and J. K. Loebbecke. 1997. Reliance on decision aids: An examination of auditors
assessment of management fraud. Auditing: A Journal of Practice & Theory 16 (2): 119.
Ely, J. W., M. L. Graber, and P. Croskerry. 2011. Checklists to reduce diagnostic errors. Academic Medicine 86 (3):
307313.
Federal Aviation Administration (FAA). 2007. Section 12: Aircraft Checklists for 14 CFR parts 121/135 iFOFSIMSF
Available at: http://fsims.faa.gov/PICDetail.aspx?docId8900.1,Vol.3,Ch32,Sec12
Fischhoff, B., P. Slovic, and S. Lichtenstein. 1978. Fault trees: Sensitivity of estimated failure probabilities to problem
representation. Journal of Experimental Psychology: Human Perception and Performance 4 (2): 330344.
Gawande, A. 2007. The checklist. New Yorker Annals of Medicine. (December 10).
Gawande, A. 2009. The Checklist Manifesto: How to Get Things Right. New York, NY: Metropolitan Books.
Glover, S. M., D. F. Prawitt, and B. C. Spilker. 1997. The influence of decision aids on user behavior: Implications for
knowledge acquisition and inappropriate reliance. Organizational Behavior and Human Decision Processes 72
(2): 232255.
Gomaa, M. I., J. E. Hunton, and J. M. Rose. 2008. The effects of control risk and litigation risk on decision aid reliance.
International Journal of Applied Decision Sciences 1 (1): 80106.
Gramling, A. A., and M. G. Watson. 2009. Analysis of peer review reports: A focus on deficiencies of the top-20
triennially inspected firms. Current Issues in Auditing 3 (2): A1A14.
Hammersley, J. S. 2011. A review and model of auditor judgments in fraud-related planning tasks. Auditing: A Journal
of Practice & Theory 30 (4): 101128.
Hammond, K., and D. Summers. 1972. Cognitive control. Psychological Review 79: 5867.
Haynes, A. B., T. G. Weiser, W. R. Berry, S. R. Lipsitz, A-H. S. Breizat, E. P. Dellinger, T. Herbosa, S. Joseph, P. L.
Kibatala, M. C. M. Lapitan, A. F. Merry, K. Moorthy, R. K. Reznick, B. Taylor, and A. A. Gawande. 2009. A
surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of
Medicine 360: 491499.
Hoffman, V. B., and M. F. Zimbelman. 2009. Do strategic reasoning and brainstorming help auditors change their
standard audit procedures in response to fraud risk? The Accounting Review 84 (3): 811837.
Hoffman, V. B., and M. F. Zimbelman. 2012. How strategic reasoning and brainstorming can help auditors detect
fraud. Current Issues in Auditing 6 (2): P25P33.
Hogan, C. E., Z. Rezaee, R. A. Riley, Jr., and U. K. Velury. 2008. Financial statement fraud: Insights from the
Academic Literature. Auditing: A Journal of Practice &Theory 27 (2): 231252.
Jamal, K. 2008. Mandatory audit of financial reporting: A failed strategy for dealing with fraud. Accounting
Perspectives 7 (2): 97110.
Jennings, M., D. C. Kneer, and P. M. J. Reckers. 1993. The significance of audit decision aids and pre-case jurists
attitudes on perceptions of audit firm culpability and liability. Contemporary Accounting Research 9 (2): 489507.
Kahneman, D. 2011. Thinking, Fast and Slow. New York, NY: Doubleday.
Kahneman, D., D. Lovallo, and O. Sibony. 2011. Before you make that big decision. Harvard Business Review (June):
5060.
Kaplan, S. E., J. H. Reneau, and S. Whitecotton. 2001. The effects of predictive ability information, locus of control,
and decision maker involvement on decision aid reliance. Journal of Behavioral Decision Making 14: 3550.
Kerr, N. L., and R. S. Tindale. 2004. Group performance and decision making. Annual Review of Psychology 55: 623
655.
Knechel, W. R., G. V. Krishnan, M. Pevzner, L. B. Shefchik, and U. K. Velury. 2013. Audit quality: Insights from the
academic literature. Auditing: A Journal of Practice & Theory 32 (S1): 385421.
KPMG. 2011. Elevating Professional Judgment in Auditing. Montvale, NJ: KPMG.
Libby, R., and P. A. Libby. 1989. Expert measurement and mechanical combination in control reliance decisions. The
Accounting Review 64 (4): 729747.
Lowe, J., P. Reckers, and S. Whitecotton. 2002. The effect of decision aid use and reliability on jurors evaluations of
auditor liability. The Accounting Review 77: 185202.
Marley, R. N. 2011. An Empirical Investigation of Decision Aids to Improve Auditor Effectiveness in Analytical Review.
Theses and Dissertations Paper 3232. Available at: http://scholarcommons.usf.edu/etd/3232
McDaniel, L. S. 1990. The effects of time pressure and audit program structure on audit performance. Journal of
Accounting Research 28 (2): 267285.
McKee, T. 2010. The cry wolf problem in current fraud auditing standards. The CPA Journal (January): 6062.
Nelson, M., and H.-T. Tan. 2005. Judgment and DM research in auditing: A task, person, and interpersonal interaction
perspective. Auditing: A Journal of Practice & Theory 24 (S): 4172.
Ng, T. B.-P., and H.-T. Tan. 2003. Effects of authoritative guidance availability and audit committee effectiveness on
auditors judgments in an auditor-client negotiation context. The Accounting Review 78 (3): 801818.
Pincus, K. V. 1989. The efficacy of a red flags questionnaire for assessing the possibility of fraud. Accounting,
Organizations and Society 14 (1/2): 153163.
PricewaterhouseCoopers LLP (PwC). 2011. Qualitative Goodwill Impairment Assessment. Available at: http://www.
pwc.com/us/en/audit-assurance-services/valuation/publications/goodwill-impairment-testing.jhtml
Pronovost P., D. Needham, S. Berenholtz, D. Sinopoli, H. Chu, S. Cosgrove, B. Sexton, R. Hyzy, R. Welsh, G. Roth, J.
Bander, J. Kepros, and C. Goeschel. 2006. An intervention to decrease catheter-related bloodstream infections
in the ICU. New England Journal of Medicine 355: 27252732.
Public Company Accounting Oversight Board (PCAOB). 2001. AT Section 101 Attest Engagements. Available at:
http://pcaobus.org/Standards/Attestation/Pages/AT101.aspx#ps-pcaob_eb7d73cb-d5be-41e4-9242-
5f204591ad07
Public Company Accounting Oversight Board (PCAOB). 2006. Testimony Concerning Sarbanes-Oxley at Four:
Protecting Investors and Strengthening the Markets. Available at: http://pcaobus.org/News/Speech/Pages/
09192006_OlsonHouseFinancialServicesCommittee.aspx
Public Company Accounting Oversight Board (PCAOB). 2012. Current Trends and Issues in Public Company
Auditing. Available at: http://pcaobus.org/News/Speech/Pages/10262012_Franzel.aspx
Public Company Accounting Oversight Board (PCAOB). 2013. In the Matter of Ernst and Young LLPs Quality Control
Remediation Submission. PCAOB Release No. 104-2013-087. Available at: http://pcaobus.org/Inspections/
Documents/05232013_EYReportStatement.pdf
Raiffa, H. 1968. Decision Analysis. Reading, MA: Addison-Wesley.
Romney, M. B., W. S. Albrecht, and D. J. Cherrington. 1980. Auditors and the detection of fraud. Journal of
Accountancy 149 (5): 6369.
Rose, A. M., and J. M. Rose. 2003. The effects of fraud risk assessments and a risk analysis decision aid on auditors
evaluation of evidence and judgment. Accounting Forum 27 (3): 312338.
Rose, J. M., B. A. McKay, C. S. Norman, and A. M. Rose. 2012. Designing decision aids to promote the development
of expertise. Journal of Information Systems 26 (1): 734.
Seow, J.-L. 2009. Cue usage in financial statement fraud risk assessments: Effects of technical knowledge and
decision aid use. Accounting and Finance 49: 183205.
Seow, P.-S. 2011. The effects of decision aid structural effectiveness on decision making outcomes. International
Journal of Accounting Information Systems 12: 4056.
Shelton, S. W., O. R. Whittington, and D. Landsittel. 2001. Auditing firms fraud risk assessment practices. Accounting
Horizons 15 (1): 1933.
Sieck, W. R., and H. R. Arkes. 2005. The recalcitrance of overconfidence and its contribution to decision aid neglect.
Journal of Behavioral Decision Making 18: 2953.
Sloman, S. A. 1996. The empirical case for two systems of reasoning. Psychological Bulletin 119 (1): 322.
Sutton, S. G., R. Young, and P. McKenzie. 1994. An analysis of potential legal liability incurred through audit expert
systems. Intelligent Systems in Accounting, Finance, and Management 4: 191204.
Trotman, K. T., P. W. Yetton, and I. R. Zimmer. 1983. Individual and group judgments of internal control systems.
Journal of Accounting Research 21 (1): 286292.
Turner, T. 2001. Controlling Pilot Error: Checklists and Compliance. New York, NY: McGraw-Hill Professional.
U.S. Nuclear Regulatory Commission (U.S. NRC). 1981. Fault Tree Handbook. Washington, DC: Systems and
Reliability Research Office of Nuclear Regulatory Research U.S. Nuclear Regulatory Commission.
Wells, J. T. 1997. Occupational Fraud and Abuse. London, U.K.: Obsidian Publishing Company.
Wheeler, P. R., and V. Arunachalam. 2008. The effects of decision aid designs on the information search strategies
and confirmation bias of tax professionals. Behavioral Research in Accounting 20: 131145.
Wilks, T. J., and M. F. Zimbelman. 2004. Decomposition of fraud-risk assessments and auditors sensitivity to fraud
cues. Contemporary Accounting Research 21 (3): 719745.
Winters, B. D., A. P. Gurses, H. Lehmann, J. B. Sexton, C. J. Rampersad, and P. J. Pronovost. 2009. Clinical review:
Checkliststranslating evidence into practice. Critical Care 13: 210.
Winters, B. D., M. S. Aswani, and P. J. Pronovost. 2011. Reducing diagnostic errors: Another role for checklists?
Academic Medicine 86 (3): 279281.
Wood, L. I. 2012. The Impact of Decision Aid Use on the Dilution Effect When Assessing Fraud. Available at: http://
www.aabri.com/manuscripts/121118.pdf
A
P
P
E
N
D
I
X
A
L
i
t
e
r
a
t
u
r
e
o
n
t
h
e
U
s
e
o
f
C
h
e
c
k
l
i
s
t
s
a
n
d
C
h
e
c
k
l
i
s
t
-
B
a
s
e
d
A
i
d
s
i
n
A
u
d
i
t
i
n
g
S
o
u
r
c
e
D
e
c
i
s
i
o
n
A
i
d
T
y
p
e
o
f
J
u
d
g
m
e
n
t
M
e
t
h
o
d
o
l
o
g
y
P
a
r
t
i
c
i
p
a
n
t
s
a
n
d
S
a
m
p
l
e
S
i
z
e
C
h
e
c
k
l
i
s
t
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
C
h
e
c
k
l
i
s
t
-
B
a
s
e
d
A
i
d
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
A
l
o
n
a
n
d
D
w
y
e
r
(
2
0
1
0
)
C
h
e
c
k
l
i
s
t
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
6
2
u
n
d
e
r
g
r
a
d
u
a
t
e
a
u
d
i
t
i
n
g
s
t
u
d
e
n
t
s
M
o
r
e
e
f
f
e
c
t
i
v
e
(
i
n
b
o
t
h
i
n
d
i
v
i
d
u
a
l
a
n
d
g
r
o
u
p
s
e
t
t
i
n
g
s
)
N
A
A
s
a
r
e
a
n
d
W
r
i
g
h
t
(
2
0
0
4
)
C
h
e
c
k
l
i
s
t
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
6
9
a
u
d
i
t
o
r
s
L
e
s
s
e
f
f
e
c
t
i
v
e
N
A
A
s
h
t
o
n
(
1
9
7
4
)
C
h
e
c
k
l
i
s
t
-
b
a
s
e
d
m
o
d
e
l
w
i
t
h
p
r
e
s
c
r
i
b
e
d
f
a
c
t
o
r
w
e
i
g
h
t
i
n
g
I
n
t
e
r
n
a
l
c
o
n
t
r
o
l
E
x
p
e
r
i
m
e
n
t
a
l
6
3
a
u
d
i
t
o
r
s
N
A
M
o
r
e
e
f
f
e
c
t
i
v
e
(
b
e
t
t
e
r
c
o
n
s
i
s
t
e
n
c
y
)
B
a
m
b
e
r
,
C
a
r
p
e
n
t
e
r
,
a
n
d
H
a
m
m
e
r
s
l
e
y
(
2
0
0
7
)
C
h
e
c
k
l
i
s
t
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
1
2
3
a
u
d
i
t
s
e
n
i
o
r
s
L
e
s
s
e
f
f
e
c
t
i
v
e
(
i
n
g
r
o
u
p
s
e
t
t
i
n
g
s
)
N
A
B
e
l
l
a
n
d
C
a
r
c
e
l
l
o
(
2
0
0
0
)
C
h
e
c
k
l
i
s
t
-
b
a
s
e
d
L
o
g
i
t
m
o
d
e
l
F
r
a
u
d
r
i
s
k
S
u
r
v
e
y
a
n
d
a
r
c
h
i
v
a
l
2
3
3
p
a
r
t
n
e
r
s
,
7
1
m
a
n
a
g
e
r
s
,
a
n
d
1
s
e
n
i
o
r
a
u
d
i
t
o
r
N
A
M
o
r
e
e
f
f
e
c
t
i
v
e
B
e
d
a
r
d
a
n
d
G
r
a
h
a
m
(
2
0
0
2
)
C
h
e
c
k
l
i
s
t
-
b
a
s
e
d
a
i
d
e
m
p
h
a
s
i
z
i
n
g
n
e
g
a
t
i
v
e
/
p
o
s
i
t
i
v
e
o
r
i
e
n
t
a
t
i
o
n
I
n
h
e
r
e
n
t
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
1
9
s
e
n
i
o
r
s
a
n
d
s
u
p
e
r
v
i
s
i
n
g
s
e
n
i
o
r
s
,
2
5
m
a
n
a
g
e
r
s
a
n
d
s
e
n
i
o
r
m
a
n
a
g
e
r
s
,
a
n
d
2
p
a
r
t
n
e
r
s
N
A
N
e
g
a
t
i
v
e
l
y
o
r
i
e
n
t
e
d
a
i
d
m
o
r
e
e
f
f
e
c
t
i
v
e
t
h
a
n
p
o
s
i
t
i
v
e
l
y
o
r
i
e
n
t
e
d
o
n
e
(
c
o
n
t
i
n
u
e
d
o
n
n
e
x
t
p
a
g
e
)
A
P
P
E
N
D
I
X
A
(
c
o
n
t
i
n
u
e
d
)
S
o
u
r
c
e
D
e
c
i
s
i
o
n
A
i
d
T
y
p
e
o
f
J
u
d
g
m
e
n
t
M
e
t
h
o
d
o
l
o
g
y
P
a
r
t
i
c
i
p
a
n
t
s
a
n
d
S
a
m
p
l
e
S
i
z
e
C
h
e
c
k
l
i
s
t
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
C
h
e
c
k
l
i
s
t
-
B
a
s
e
d
A
i
d
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
B
e
l
l
,
B
e
d
a
r
d
,
J
o
h
n
s
t
o
n
e
,
a
n
d
S
m
i
t
h
(
2
0
0
2
)
C
h
e
c
k
l
i
s
t
-
b
a
s
e
d
e
l
e
c
t
r
o
n
i
c
a
i
d
C
l
i
e
n
t
a
c
c
e
p
t
a
n
c
e
F
i
e
l
d
s
t
u
d
y
(
K
P
M
G
)
N
A
N
A
N
A
B
l
o
c
h
e
r
,
E
s
p
o
s
i
t
o
,
a
n
d
W
i
l
l
i
n
g
h
a
m
(
1
9
8
3
)
C
h
e
c
k
l
i
s
t
A
n
a
l
y
t
i
c
a
l
r
e
v
i
e
w
E
x
p
e
r
i
m
e
n
t
a
l
4
4
m
a
n
a
g
e
m
e
n
t
a
n
d
s
e
n
i
o
r
-
l
e
v
e
l
a
u
d
i
t
o
r
s
M
i
x
e
d
r
e
s
u
l
t
,
p
l
a
n
n
e
d
m
o
r
e
o
f
b
o
t
h
a
n
a
l
y
t
i
c
a
l
a
n
d
t
e
s
t
s
o
f
d
e
t
a
i
l
s
p
r
o
c
e
d
u
r
e
s
N
A
B
o
a
t
s
m
a
n
,
M
o
e
c
k
e
l
,
a
n
d
P
e
i
(
1
9
9
7
)
C
h
e
c
k
l
i
s
t
-
b
a
s
e
d
a
i
d
p
r
o
v
i
d
i
n
g
c
u
e
a
g
g
r
e
g
a
t
i
o
n
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
1
1
8
a
u
d
i
t
s
e
n
i
o
r
s
N
A
I
n
t
e
n
t
i
o
n
a
l
s
h
i
f
t
i
n
g
n
a
l
j
u
d
g
m
e
n
t
a
w
a
y
f
r
o
m
a
i
d
s
p
r
e
d
i
c
t
i
o
n
a
n
d
i
g
n
o
r
i
n
g
a
i
d
w
h
e
n
i
t
s
p
r
e
d
i
c
t
i
o
n
d
o
e
s
n
o
t
s
u
p
p
o
r
t
i
n
i
t
i
a
l
j
u
d
g
m
e
n
t
m
a
y
u
n
d
e
r
m
i
n
e
a
i
d
u
s
e
B
o
n
n
e
r
,
L
i
b
b
y
,
a
n
d
N
e
l
s
o
n
(
1
9
9
6
)
C
h
e
c
k
l
i
s
t
a
n
d
d
e
c
o
m
p
o
s
i
t
i
o
n
-
a
n
d
-
m
e
c
h
a
n
i
c
a
l
-
a
g
g
r
e
g
a
t
i
o
n
a
i
d
E
r
r
o
r
f
r
e
q
u
e
n
c
y
p
r
o
b
a
b
i
l
i
t
y
E
x
p
e
r
i
m
e
n
t
a
l
1
0
5
a
u
d
i
t
o
r
s
M
a
r
g
i
n
a
l
l
y
m
o
r
e
e
f
f
e
c
t
i
v
e
M
o
r
e
e
f
f
e
c
t
i
v
e
B
o
r
i
t
z
(
1
9
8
5
)
L
i
n
e
a
r
a
n
d
h
i
e
r
a
r
c
h
i
c
a
l
l
y
s
t
r
u
c
t
u
r
e
d
c
h
e
c
k
l
i
s
t
s
A
u
d
i
t
p
l
a
n
n
i
n
g
E
x
p
e
r
i
m
e
n
t
a
l
4
0
a
u
d
i
t
o
r
s
I
n
m
o
s
t
c
a
s
e
s
s
t
r
u
c
t
u
r
e
d
l
i
s
t
s
p
r
o
v
i
d
e
f
o
r
m
o
r
e
u
n
i
f
o
r
m
i
t
y
t
h
a
n
l
i
n
e
a
r
o
n
e
s
N
A
B
u
t
l
e
r
(
1
9
8
5
)
S
e
l
f
-
e
l
i
c
i
t
a
t
i
o
n
c
h
e
c
k
l
i
s
t
T
e
s
t
o
f
d
e
t
a
i
l
s
s
a
m
p
l
i
n
g
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
2
3
a
u
d
i
t
o
r
s
M
o
r
e
e
f
f
e
c
t
i
v
e
N
A
(
c
o
n
t
i
n
u
e
d
o
n
n
e
x
t
p
a
g
e
)
A
P
P
E
N
D
I
X
A
(
c
o
n
t
i
n
u
e
d
)
S
o
u
r
c
e
D
e
c
i
s
i
o
n
A
i
d
T
y
p
e
o
f
J
u
d
g
m
e
n
t
M
e
t
h
o
d
o
l
o
g
y
P
a
r
t
i
c
i
p
a
n
t
s
a
n
d
S
a
m
p
l
e
S
i
z
e
C
h
e
c
k
l
i
s
t
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
C
h
e
c
k
l
i
s
t
-
B
a
s
e
d
A
i
d
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
C
o
w
p
e
r
t
h
w
a
i
t
e
(
2
0
1
2
)
G
e
n
e
r
i
c
a
n
d
c
u
s
t
o
m
i
z
e
d
c
h
e
c
k
l
i
s
t
s
V
a
r
i
o
u
s
a
u
d
i
t
j
u
d
g
m
e
n
t
s
C
o
m
m
e
n
t
a
r
y
N
A
P
o
t
e
n
t
i
a
l
a
d
v
a
n
t
a
g
e
s
o
f
c
u
s
t
o
m
i
z
e
d
c
h
e
c
k
l
i
s
t
s
t
o
b
e
i
n
v
e
s
t
i
g
a
t
e
d
N
A
E
i
n
i
n
g
,
J
o
n
e
s
,
a
n
d
L
o
e
b
b
e
c
k
e
(
1
9
9
7
)
C
h
e
c
k
l
i
s
t
,
c
h
e
c
k
l
i
s
t
-
b
a
s
e
d
L
o
g
i
t
m
o
d
e
l
,
a
n
d
c
h
e
c
k
l
i
s
t
-
b
a
s
e
d
e
x
p
e
r
t
s
y
s
t
e
m
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
9
6
a
u
d
i
t
o
r
s
N
o
d
i
f
f
e
r
e
n
c
e
M
o
r
e
e
f
f
e
c
t
i
v
e
L
i
b
b
y
a
n
d
L
i
b
b
y
(
1
9
8
9
)
C
o
m
p
o
n
e
n
t
j
u
d
g
m
e
n
t
a
g
g
r
e
g
a
t
i
o
n
a
i
d
I
n
t
e
r
n
a
l
c
o
n
t
r
o
l
E
x
p
e
r
i
m
e
n
t
a
l
8
9
s
e
n
i
o
r
a
u
d
i
t
o
r
s
N
A
M
o
r
e
e
f
f
e
c
t
i
v
e
M
a
r
l
e
y
(
2
0
1
1
)
G
r
a
p
h
i
c
a
l
d
e
p
i
c
t
i
o
n
o
f
c
o
m
m
o
n
a
c
c
o
u
n
t
i
n
g
r
e
l
a
t
i
o
n
s
h
i
p
s
a
n
d
a
i
d
f
o
r
a
u
t
o
m
a
t
i
c
r
e
c
a
l
l
o
f
a
u
d
i
t
o
r
-
i
d
e
n
t
i
e
d
c
u
e
s
A
n
a
l
y
t
i
c
a
l
r
e
v
i
e
w
E
x
p
e
r
i
m
e
n
t
a
l
1
2
9
a
c
c
o
u
n
t
i
n
g
s
t
u
d
e
n
t
s
N
A
M
o
r
e
e
f
f
e
c
t
i
v
e
M
u
e
l
l
e
r
a
n
d
A
n
d
e
r
s
o
n
(
2
0
0
2
)
C
h
e
c
k
l
i
s
t
w
i
t
h
e
i
t
h
e
r
e
l
i
m
i
n
a
t
i
o
n
o
r
i
n
c
l
u
s
i
o
n
g
o
a
l
f
r
a
m
e
A
n
a
l
y
t
i
c
a
l
r
e
v
i
e
w
E
x
p
e
r
i
m
e
n
t
a
l
1
9
a
u
d
i
t
o
r
s
,
2
0
s
e
n
i
o
r
a
u
d
i
t
o
r
s
,
2
0
a
u
d
i
t
m
a
n
a
g
e
r
s
,
2
a
u
d
i
t
p
a
r
t
n
e
r
s
,
a
n
d
4
s
u
b
j
e
c
t
s
w
i
t
h
u
n
k
n
o
w
n
r
a
n
k
G
o
a
l
f
r
a
m
e
h
a
s
b
e
a
r
i
n
g
o
n
c
h
e
c
k
l
i
s
t
a
p
p
l
i
c
a
t
i
o
n
r
e
s
u
l
t
s
N
A
(
c
o
n
t
i
n
u
e
d
o
n
n
e
x
t
p
a
g
e
)
A
P
P
E
N
D
I
X
A
(
c
o
n
t
i
n
u
e
d
)
S
o
u
r
c
e
D
e
c
i
s
i
o
n
A
i
d
T
y
p
e
o
f
J
u
d
g
m
e
n
t
M
e
t
h
o
d
o
l
o
g
y
P
a
r
t
i
c
i
p
a
n
t
s
a
n
d
S
a
m
p
l
e
S
i
z
e
C
h
e
c
k
l
i
s
t
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
C
h
e
c
k
l
i
s
t
-
B
a
s
e
d
A
i
d
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
P
i
n
c
u
s
(
1
9
8
9
)
C
h
e
c
k
l
i
s
t
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
1
3
7
a
u
d
i
t
o
r
s
L
e
s
s
e
f
f
e
c
t
i
v
e
N
A
R
o
s
e
a
n
d
R
o
s
e
(
2
0
0
3
)
C
u
e
-
a
g
g
r
e
g
a
t
i
o
n
a
i
d
(
w
i
t
h
n
o
p
r
e
d
e
n
e
d
c
u
e
s
l
i
s
t
)
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
7
4
a
u
d
i
t
o
r
s
N
A
M
o
r
e
e
f
f
e
c
t
i
v
e
(
l
e
s
s
r
e
c
e
n
c
y
b
i
a
s
)
R
o
s
e
e
t
a
l
.
(
2
0
1
2
)
C
h
e
c
k
l
i
s
t
o
r
g
a
n
i
z
e
d
a
l
o
n
g
f
r
a
u
d
t
r
i
a
n
g
l
e
o
r
a
l
o
n
g
a
g
g
r
e
g
a
t
e
e
x
p
e
r
t
k
n
o
w
l
e
d
g
e
s
t
r
u
c
t
u
r
e
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
1
1
5
s
e
n
i
o
r
-
l
e
v
e
l
a
c
c
o
u
n
t
i
n
g
s
t
u
d
e
n
t
s
K
n
o
w
l
e
d
g
e
s
t
r
u
c
t
u
r
e
c
h
e
c
k
l
i
s
t
s
m
o
r
e
e
f
f
e
c
t
i
v
e
t
h
a
n
f
r
a
u
d
t
r
i
a
n
g
l
e
o
n
e
s
(
f
o
r
l
e
a
r
n
i
n
g
)
N
A
S
e
o
w
(
2
0
0
9
)
C
h
e
c
k
l
i
s
t
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
3
2
c
o
r
p
o
r
a
t
e
d
i
r
e
c
t
o
r
s
,
1
1
9
s
t
u
d
e
n
t
s
L
e
s
s
e
f
f
e
c
t
i
v
e
f
o
r
d
i
r
e
c
t
o
r
s
,
m
o
r
e
e
f
f
e
c
t
i
v
e
f
o
r
s
t
u
d
e
n
t
s
N
A
S
e
o
w
(
2
0
1
1
)
C
h
e
c
k
l
i
s
t
-
b
a
s
e
d
a
i
d
s
(
s
t
r
u
c
t
u
r
a
l
l
y
r
e
s
t
r
i
c
t
i
v
e
o
r
n
o
n
r
e
s
t
r
i
c
t
i
v
e
)
I
n
t
e
r
n
a
l
c
o
n
t
r
o
l
E
x
p
e
r
i
m
e
n
t
a
l
9
4
s
t
u
d
e
n
t
s
N
A
L
e
s
s
s
t
r
u
c
t
u
r
a
l
l
y
r
e
s
t
r
i
c
t
i
v
e
a
i
d
m
o
r
e
e
f
c
i
e
n
t
t
h
a
n
m
o
r
e
r
e
s
t
r
i
c
t
i
v
e
o
n
e
T
r
o
t
m
a
n
,
Y
e
t
t
o
n
,
a
n
d
Z
i
m
m
e
r
(
1
9
8
3
)
C
h
e
c
k
l
i
s
t
I
n
t
e
r
n
a
l
c
o
n
t
r
o
l
E
x
p
e
r
i
m
e
n
t
a
l
1
0
5
a
c
c
o
u
n
t
i
n
g
m
a
j
o
r
s
t
u
d
e
n
t
s
I
n
t
e
r
a
c
t
i
v
e
g
r
o
u
p
s
a
n
d
e
q
u
i
v
a
l
e
n
t
c
o
m
p
o
s
i
t
e
s
p
e
r
f
o
r
m
e
q
u
a
l
l
y
w
h
e
n
u
s
i
n
g
c
h
e
c
k
l
i
s
t
s
N
A
(
c
o
n
t
i
n
u
e
d
o
n
n
e
x
t
p
a
g
e
)
A
P
P
E
N
D
I
X
A
(
c
o
n
t
i
n
u
e
d
)
S
o
u
r
c
e
D
e
c
i
s
i
o
n
A
i
d
T
y
p
e
o
f
J
u
d
g
m
e
n
t
M
e
t
h
o
d
o
l
o
g
y
P
a
r
t
i
c
i
p
a
n
t
s
a
n
d
S
a
m
p
l
e
S
i
z
e
C
h
e
c
k
l
i
s
t
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
C
h
e
c
k
l
i
s
t
-
B
a
s
e
d
A
i
d
v
e
r
s
u
s
U
n
a
i
d
e
d
J
u
d
g
m
e
n
t
P
e
r
f
o
r
m
a
n
c
e
W
h
e
e
l
e
r
a
n
d
A
r
u
n
a
c
h
a
l
a
m
(
2
0
0
8
)
J
u
s
t
i
c
a
t
i
o
n
r
e
q
u
i
r
e
m
e
n
t
a
i
d
a
n
d
c
h
e
c
k
l
i
s
t
T
a
x
r
e
s
e
a
r
c
h
E
x
p
e
r
i
m
e
n
t
a
l
1
4
2
t
a
x
p
r
o
f
e
s
s
i
o
n
a
l
s
L
e
s
s
e
f
f
e
c
t
i
v
e
(
m
o
r
e
c
o
n
r
m
a
t
i
o
n
b
i
a
s
)
M
o
r
e
e
f
f
e
c
t
i
v
e
(
l
e
s
s
c
o
n
r
m
a
t
i
o
n
b
i
a
s
)
W
i
l
k
s
a
n
d
Z
i
m
b
e
l
m
a
n
(
2
0
0
4
)
L
i
n
e
a
r
c
h
e
c
k
l
i
s
t
a
n
d
c
h
e
c
k
l
i
s
t
o
r
g
a
n
i
z
e
d
a
l
o
n
g
f
r
a
u
d
t
r
i
a
n
g
l
e
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
5
2
a
u
d
i
t
m
a
n
a
g
e
r
s
C
h
e
c
k
l
i
s
t
o
r
g
a
n
i
z
e
d
a
l
o
n
g
f
r
a
u
d
t
r
i
a
n
g
l
e
m
o
r
e
e
f
f
e
c
t
i
v
e
t
h
a
n
l
i
n
e
a
r
o
n
e
N
A
W
o
o
d
(
2
0
1
2
)
C
h
e
c
k
l
i
s
t
F
r
a
u
d
r
i
s
k
E
x
p
e
r
i
m
e
n
t
a
l
1
4
0
a
u
d
i
t
s
e
n
i
o
r
s
L
e
s
s
e
f
f
e
c
t
i
v
e
(
d
u
e
t
o
d
i
l
u
t
i
o
n
b
i
a
s
)
N
A

Ciia 50741

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Ciia 50741

Transféré par

Droits d'auteur :

Formats disponibles

Current Issues in Auditing American Accounting Association

Volume 8, Issue 1 DOI: 10.2308/ciia-50741

Vous aimerez peut-être aussi