Auditing: A Journal of Practice & Theory American Accounting Association

Vol. 30, No. 3 DOI: 10.2308/ajpt-10054

August 2011
pp. 211–224

Internal Auditors’ Fraud Judgments: The

Benefits of Brainstorming in Groups
Tina D. Carpenter, Jane L. Reimers, and Phillip Z. Fretwell
SUMMARY: Recent fraud scandals have encouraged actions by standard-setters to
improve both corporate governance among firms and auditors’ fraud investigations.
Internal auditors are now viewed as playing an important role in reducing fraudulent
financial reporting. Although brainstorming is not required by internal auditors,
researchers and leaders in the profession suggest that it may be helpful to internal
auditors in assessing and identifying risks. In this study, we investigate whether the
group interaction associated with brainstorming is necessary to reap the benefits of
brainstorming for internal auditors’ fraud judgments. Guided by psychology theory on
cognitive load, we also examine whether this group interaction can reduce a response
mode bias that auditors have exhibited when assessing risk. Consistent with prior
research on external auditors, we find that internal auditors who brainstorm in groups
identify fewer fraud risks (i.e., quantity) than nominal groups of individual auditors who
brainstorm alone, but brainstorming groups identify more quality fraud risks than nominal
groups. Further, we find that auditors who assess risk qualitatively generally provide
higher fraud risk assessments than those auditors who assess risk quantitatively.
However, after group brainstorming this bias is reduced.
Keywords: fraud risk assessments; brainstorming; response mode bias; group
interaction; internal audit.
Data Availability: Contact the authors.

Tina D. Carpenter is an Associate Professor at The University of Georgia, Jane L. Reimers is a Professor at
Rollins College, and Phillip Z. Fretwell is a Managing Director at Protiviti, Orlando, FL.

We are grateful for the financial support provided by a grant from the Institute of Internal Auditors Research Foundation
and for the advice and guidance provided by Larry Rittenberg. We also thank Michael Bamber, Ann Backof, Al Bathke,
Mark Beasley, Joe Brazel, Margaret Christ, Greg Gerard, Audrey Gramling, Jessen Hobson, Steve Salterio, and
workshop participants at Florida State University, North Carolina State University, and the American Accounting
Association Annual Meeting for their helpful comments and suggestions. We are especially indebted to Protiviti and The
Institute of Internal Auditors for their help in providing participants for the experiment. We also appreciate the financial
assistance provided by a Terry Sanford Research Grant from The University of Georgia.
Editor’s note: Accepted by Ken Trotman.

Submitted: September 2008

Accepted: June 2010
Published Online: August 2011

211
212 Carpenter, Reimers, and Fretwell

INTRODUCTION

R
ecent financial reporting scandals have prompted actions by standard-setters directed at
improving both auditors’ investigation of fraud and corporate governance. In the past,
responses to these types of scandals did not focus on the role of internal auditors.
However, standard-setters, external auditors, and audit committees are now looking to internal
auditors as part of the solution to a perceived breakdown in financial reporting and managements’
ethical behavior (Institute of Internal Auditors [IIA] 2003, 2006; Gramling et al. 2004;
PricewaterhouseCoopers 2009). Although brainstorming is not required by internal auditors,
researchers and leaders in the profession suggest that it may be helpful to internal auditors in
assessing and identifying risks (Kinney 2003; IIA 2009). In this study, we investigate internal
auditors’ fraud judgments to determine the effects of brainstorming1 type—group versus
individual—on internal auditors’ fraud judgments and whether group brainstorming can reduce a
response mode bias that may arise when auditors assess fraud risk either quantitatively or
qualitatively.
This study makes the following contributions. First, this study extends the work of Carpenter
(2007), who examined SAS No. 99 brainstorming among external auditors, to internal auditors.2
Internal auditors have been shown to provide judgments that are different from external auditors
(Church and Schneider 1995), so it is important to understand whether interacting groups of internal
auditors overcome limitations in individual auditors’ judgments. Further, the internal-auditor
participants in this study differ from the external auditors Carpenter (2007) studied in several
important ways: (1) while researchers and practitioners have suggested that brainstorming may be
helpful to internal auditors (e.g., Kinney 2003; IIA 2009), it is not required in practice; (2) internal
auditors’ risk identification and assessments are more comprehensive (e.g., operational risks,
financial risks, and compliance with laws and regulations risks, in addition to fraud-related risks)
and this broader focus likely provides less emphasis on fraud risks for internal auditors; and (3)
internal auditors can work in teams, but they are not required to, and therefore often work as
individuals—and if they do work in teams, then they are often homogenous teams of internal
auditors at similar experience levels sharing varied experiences over various risk areas.3 Therefore,
we investigate whether internal auditors benefit from brainstorming, and, if so, whether the benefits
of brainstorming could be realized without group interaction.
Results from our experiment suggest that homogenous brainstorming groups of internal
auditors identify fewer fraud risks (i.e., quantity) than a collection of individual internal auditors
(i.e., nominal group), but they identify more quality fraud risks (i.e., actual frauds identified by the

1
Brainstorming often implies a group interaction, but it can be defined as the generation of ideas by one or more
individuals (Dugosh et al. 2000). In this paper, we use the term ‘‘brainstorming’’ to mean the generation of ideas
by one or more individuals given a specific task to ‘‘brainstorm, listing all of the risks that are relevant’’ for a
given case. Further, brainstorming sessions as mandated by SAS No. 99 for external auditors, include risk
generation and an overall risk assessment. If internal auditors were to implement brainstorming, then it would
likely be similar to external auditors’ implementation. Thus, our group brainstorming includes both idea
generation as well group risk assessment.
2
Internal auditors are participants in our study for several important reasons. First, internal auditors are now being
considered an integral part of the corporate governance structure designed to reduce fraudulent financial reporting
(IIA 2003, 2006; Gramling et al. 2004), and current literature provides little guidance on internal auditors’
judgments in this new environment (Gramling et al. 2004). Second, while the Standards for the Professional
Practice of Internal Auditing suggests that internal auditors should be alert to the possibility of fraudulent
financial reporting (IIA 1998), they are not mandated to brainstorm about fraud as are external auditors under SAS
No. 99.
3
Based on discussions with internal audit directors in practice, internal audit teams are different from external audit
teams in practice. They can vary from one internal auditor working individually, usually an internal audit director,
to teams of internal auditors that are often structured as homogenous teams made up of auditors at similar levels
with diverse experiences, rather than being structured as a hierarchy as is typical for external auditors.

Auditing: A Journal of Practice & Theory

August 2011
Internal Auditors’ Fraud Judgments: The Benefits of Brainstorming in Groups 213

Securities and Exchange Commission [SEC] for this case) than the nominal groups. These findings
suggest that it is the group interaction component of group brainstorming rather than the act of
brainstorming itself (i.e., the generation of ideas) that provides this benefit.
Our study provides a second contribution by investigating the joint effects of brainstorming
and response mode—quantitative versus qualitative—on internal auditors’ risk assessments.
Specifically, we examine whether internal auditors are subject to a response mode bias and, if so,
whether that bias can be reduced by the group interaction associated with brainstorming. While
other studies have documented a response mode bias with external auditors (Reimers et al. 1993),
none has, to our knowledge, addressed a possible reduction in this bias through group interaction.
Reimers et al. (1993) found that external auditors assessing risk qualitatively provided higher risk
assessments than those who assessed risk quantitatively. However, their judgments reflect a bias
because these external auditors assessed risk on the same case materials with the difference isolated
only by the response mode that each condition used to assess the risk. Our review of documentation
of risk assessments from national accounting firms and within companies and our consultation with
practice partners provide anecdotal evidence that internal auditors commonly use quantitative risk
assessments (i.e., numbers) as inputs into their computerized models, rather than qualitative risk
assessments more commonly used by external auditors.4 It has been suggested that improving
internal auditors’ risk assessments is increasingly important (PricewaterhouseCoopers 2009); that
internal auditors are subject to potential biases when assessing risks (Kinney 2003); and that
brainstorming may remind auditors to overcome biases in their fraud judgments (AICPA 2003).
Based on psychology theory about cognitive load, we predict that the group interaction
associated with brainstorming can reduce a response mode bias associated with making fraud risk
assessments qualitatively versus quantitatively. Results of our study suggest that internal auditors
who assessed risk qualitatively provided higher fraud risk assessments than did those internal
auditors who assessed risk quantitatively, consistent with prior research on external auditors.
However, after group brainstorming, there was no difference between auditors’ qualitative and
quantitative risk assessments. Thus, the bias was reduced.

RELEVANT LITERATURE AND HYPOTHESES

Advantages and Disadvantages of Brainstorming

Brainstorming has become a topic of great interest to auditors since the passage of SAS No. 99,
which requires external auditors to brainstorm (AICPA 2002), and it has recently received significant
attention from the internal audit profession as well (IIA 2009). Although the main objective of
brainstorming is to generate ideas about fraud, it is unclear how the brainstorming session is to be
conducted (AICPA 2003). Since internal auditors often work as individuals, if internal auditors could
maximize their generation of ideas by brainstorming individually without incurring the costs of
interacting in a group, then this could enhance their efficiency and effectiveness.
Osborn (1957) suggested that there are benefits of brainstorming in groups, however, much of
the prior research in psychology suggests that brainstorming groups (i.e., groups that interact in open
discussion) provide fewer ideas (i.e., quantity) than nominal groups (i.e., a collection of the unique
ideas of individuals who do not interact), with the majority of studies documenting process losses
outweighing process gains when the interacting group is compared to a collection of individuals
(Diehl and Stroebe 1987; Gallupe et al. 1991; Mullen et al. 1991; Nunamaker et al. 1991; Dennis and

4
This project was sponsored by the Institute of Internal Auditors Research Foundation as a collaborative effort with
academics and practice; thus, the research questions and design are suited to address important issues for the
profession. Internal audit partners were particularly interested in the response mode bias, as they are continuously
seeking to improve their risk assessments and risk identification.

Auditing: A Journal of Practice & Theory

August 2011
214 Carpenter, Reimers, and Fretwell

Valacich 1993). Prior research in accounting has also provided evidence that brainstorming groups
of external auditors provide fewer ideas (i.e., quantity) than nominal groups of individuals who do
not interact (Carpenter 2007; Hunton and Gold 2010). These studies predicted and found that
brainstorming groups of external auditors, like students examined in psychology studies, had process
losses attributed to production blocking (i.e., only one member can speak at a time) and social loafing
(i.e., reduced effort by relying on others on the team) as these are general features of groups
regardless of its members. Because these general features are also part of internal audit brainstorming
groups, we predict that brainstorming groups will generate fewer fraud risks (i.e., quantity) than
nominal groups formed as a collection of individual participants’ risks. Formally stated:
H1a: Brainstorming groups will identify fewer fraud risks than nominal groups.
Although Carpenter (2007) found that interacting groups of external auditors did not generate
as many ideas (i.e., quantity) as did the nominal groups of individuals, she predicted and found that
brainstorming groups generated more quality fraud ideas (i.e., relevant fraud risks) than those
generated by the individuals in the group. Hoffman and Zimbelman (2009) also documented
benefits to group brainstorming, finding that external auditors who interacted in a group were more
likely to effectively modify their audit procedures than individual auditors working independently.
Trotman et al. (2009) and Lynch et al. (2009) suggest that variations in the conduct of the
brainstorming can improve the groups’ identification of quality ideas.
Carpenter (2007) predicted that the process gains would outweigh process losses for
hierarchical teams of external auditors for the quality of ideas (i.e., actual frauds identified by the
SEC for the case materials) because of external auditors’ focus on fraud detection, their experience
and knowledge with the task, and the ability of these teams to recognize the manager as the best
member of the team because the manager is the most senior member of the team and likely exerts
the most influence. Internal auditors share a similar experience and knowledge of fraud with
external auditors; however, internal audit teams are often homogenous, rather than hierarchical,
making it more difficult to identify the best member. Further, in some situations, internal auditors
have been shown to provide different judgments from those of external auditors (Church and
Schneider 1995). Thus, it is an empirical question as to whether the ‘‘quality of ideas’’ results found
with external auditors in Carpenter (2007) will generalize to internal auditors. Since internal
auditors often work alone, it is important to understand if the benefits of brainstorming, found to be
driven by managers in Carpenter (2007), can be achieved by internal auditors of similar years in
experience on their own (i.e., generation of ideas), or whether it is necessary for these internal
auditors to interact in a group for the benefits to be realized. If internal auditors engage in group
interaction, then the groups are often more homogenously comprised of members with similar
levels of experience (in terms of years) but varying backgrounds and perspectives. Research in
psychology suggests that communication among a diverse group with varying perspectives can
create synergies, enhancing the quality of the interaction and allowing each member of the group to
build on the unique quality ideas of others, so that the process gains ultimately outweigh the process
losses (Osborn 1957; Dennis and Valacich 1993). Based on this discussion, we predict that
brainstorming groups will identify more quality ideas than nominal groups. Formally stated:
H1b: Brainstorming groups will identify more quality fraud risks than nominal groups.

Biases Related to Group Interaction

While brainstorming may not have been designed by Osborn (1957) to reduce biases, standard-
setters suggest that brainstorming will encourage a questioning mind that will remind auditors of
the need to overcome biases in their fraud judgments (AICPA 2003). Even though standard-setters

Auditing: A Journal of Practice & Theory

August 2011
Internal Auditors’ Fraud Judgments: The Benefits of Brainstorming in Groups 215

and researchers suggest that brainstorming can improve auditors’ fraud judgments (AICPA 2002;
Kinney 2003; Carpenter 2007), whether brainstorming reduces biases that may be caused by the
way risk assessments are made has not been tested. In practice, brainstorming is considered a group
activity with substantial interaction. While a few studies in the psychology literature related to
group interaction—not specifically brainstorming—have found that groups are susceptible to biases
like groupthink, which occurs when group pressure leads to a reduction in individual mental
efficiency (Janis 1972, 1982), most prior research has found that groups can reduce biases that are
exhibited by individuals (Sniezek and Henry 1989; Levine and Moreland 1990). For example,
Sniezek and Henry (1989) examined the relative accuracy of group and individual risk judgments
and found that under-estimation and over-estimation biases observed with individuals are reduced
through group interaction. Auditing research has also shown that interacting groups of auditors are
able to perform better than individuals (Trotman and Yetton 1985; Trotman 1985); that groups
were more likely than individuals to increase fraud risk assessments when fraud was present
(Reckers and Schultz 1993); that groups had higher recognition accuracy (Johnson 1994), and had
higher consensus and more complete problem analysis than individuals (Bamber et al. 1996).
However, to our knowledge, no prior research in auditing has examined whether the group
interaction associated with brainstorming can reduce individual biases. While group interaction can
reduce efficiency and performance; it has been shown more often to improve performance. Thus, it
is an open question as to whether group brainstorming can reduce individual biases as suggested by
standard-setters (AICPA 2003).

Response Mode Bias

We investigate the response mode bias in a brainstorming setting. Anecdotal evidence suggests
that internal auditors often use quantitative scales, rather than the qualitative scales more often used
by external auditors. Two common types of response modes are verbal (qualitative) and numerical
(quantitative). Reimers et al. (1993) studied the effect of response mode on external auditors’ risk
assessments and found that auditors responding qualitatively perceived the risk levels to be higher
than those responding quantitatively based on the same case, thus reflecting a bias. The
performance implications of this response mode difference on risk assessment accuracy were not
determinable. However, the difference in assessment did have a significant effect on the extent of
testing required in the audit plan; qualitative assessments translated into more audit hours than
quantitative assessments.
Psychology research on cognitive load suggests that group interaction can lead to a reduction in
biases (Petty et al. 1980; Kirschner et al. 2008, 2009). Petty et al. (1980) found that individuals
diffuse the responsibility of cognitive tasks among group members, and Kirschner et al. (2008,
2009) found that cognitive load is shared among group members, making it possible for group
members to process information more deeply and construct higher quality schemas in their long-
term memory than individuals working alone.
In an accounting task, Dusenbury and Reimers (2010) investigated the implications of requiring
a specific response mode. They hypothesized that providing quantitative assessments would be more
cognitively demanding (i.e., heavier cognitive load), so that quantitative responders would have less
excess mental capacity for remembering facts from the case. The results showed that participants
who gave qualitative assessments recalled more information from the experimental case than did the
participants who made quantitative assessments. These results support the idea that providing
quantitative assessments is associated with a heavier cognitive load than providing qualitative
assessments. If the cognitive demands of expressing risks quantitatively are greater than those of
expressing risks qualitatively, then brainstorming groups might be able to share those demands, thus
reducing the potential effect of cognitive load on making quantitative risk assessments. Given these

Auditing: A Journal of Practice & Theory

August 2011
216 Carpenter, Reimers, and Fretwell

findings, we predict that any difference due to response mode in final risk assessments will be smaller
for the groups than for individuals. That is, if pre-brainstorming risk assessments demonstrate the
expected bias (qualitative assessments are higher than quantitative assessments), then this difference
will be reduced by brainstorming in groups but not reduced by individual brainstorming. Formally
stated:
H2: Group interaction will reduce any bias associated with response mode.

METHOD
One hundred sixty-two internal auditors with an average of 6.33 years of auditing experience
participated in the experiment. The majority of these auditors were taking part in firm-wide training
for an international internal audit firm and some were attendees of an IIA’s chapter meeting,
representing a variety of other firms.5
Our experimental design for testing H1a and H1b is a 2 3 2 (brainstorming type 3 response
mode) between-participants design, and we examine quantity and quality of fraud risks for
brainstorming groups and brainstorming individuals (i.e., nominal group). H2 predicts that group
interaction will reduce any bias associated with response mode. That is, if pre-brainstorming risk
assessments demonstrate the expected bias (qualitative higher than quantitative assessments), then
this difference will be reduced by brainstorming in groups, but not reduced by individual
brainstorming. To test this prediction, we examine individual auditor judgments before and after the
brainstorm type (individual brainstorm or group brainstorm) treatment is applied. Therefore, we
employ a 2 3 2 3 2 (brainstorming type 3 response mode 3 shift) design, where the first two
variables are manipulated between-participants and the third is varied within-participants. We
examine the pre- and post-fraud risk assessments of the individual auditors to examine the response
mode bias and whether it is reduced as a result of the brainstorm type treatment. Further, to separate
the effects of group interaction and brainstorming (i.e., idea generation by an individual or group) in
reducing this bias, we also have a control group of individual auditors who perform fraud-related
tasks without brainstorming (i.e., without generating fraud ideas). This control group did not
brainstorm or list risks. Instead, these individual participants performed two tasks, completing a
psychology-based questionnaire developed by Hurtt (2010), and a fraud-related memory task that
took 20 minutes, the same amount of time that the other conditions were given to brainstorm. These
tasks were used to keep the participants thinking about fraud risk and the case materials but not
brainstorming (i.e., not generating risk ideas). Participants were randomly assigned to conditions.
The case used in the experimental materials contained fraud in the financial statements as
issued and detected by the SEC and cited in an Accounting and Auditing Enforcement Release
(AAER). A brief narrative followed by a set of financial statements with related notes was provided
to participants, along with common-sized balance sheets, income statements and selected financial
ratios.6 The experiment had three parts.
Part I required all participants to read and understand financial statements of the real company
and to provide initial risk assessments based on these materials. This part introduced the response
mode independent variable (quantitative, qualitative), which was varied between-participants.
Participants were asked to provide risk assessments on the scales shown in Appendix A. To enable

5
One or more of the researchers was present at each of the four experimental sessions to ensure that participants
followed all instructions. The results in this study were not affected by experimental session location or firm.
6
The case was adapted from Carpenter (2007). All experimental and case materials were reviewed by two internal
audit senior partners from a large international internal audit firm and by members of the IIA Research
Foundation. Thus, the materials asked internal auditors to identify risks and assess risks in this experiment as they
do in practice. Additionally, all materials were pilot-tested.

Auditing: A Journal of Practice & Theory

August 2011
Internal Auditors’ Fraud Judgments: The Benefits of Brainstorming in Groups 217

a comparison of the two types of risk assessments, we used matching scales from 0 to 6 so that we
could easily translate the qualitative judgments into quantitative values for comparison. Reimers et
al. (1993) used a variety of methods to translate quantitative and qualitative judgments to a single
measure for comparison, and none of their results were affected by the chosen method. In our case,
the construction of our scales introduces a bias against finding any differences between quantitative
and qualitative risk assessments, which thus provides a more conservative test. Participants were
not allowed to use reference materials or confer with one another while completing this part, and the
experimental materials were gathered from participants after they completed Part I, which was
restricted to 30 minutes.
Part II, which began immediately after the completion of Part I, introduced the two different
brainstorming types, group brainstorm or individual brainstorm. For the first condition, group
brainstorm, a group was randomly formed with three auditors with the same experience level. They
sat face-to-face around a table and were asked to brainstorm and list all risks, including operational
risks, financial reporting risks, fraud risks, and compliance with laws and regulation risks.7 In the
second condition, individual brainstorm, the instructions for identifying and listing risks were the
same as those provided to individuals in the group brainstorm condition. The instructions
specifically told each individual participant to brainstorm, listing all of the risks that are relevant to
the client’s financial statements provided in the case materials. For the group brainstorm condition,
to ensure a group process with a single solution, individual participants assigned to the group
brainstorm condition did not record their own risks or assessments; rather a designated recorder
was randomly selected to record the group’s conclusions. Each group produced a single list of risks,
just as the participants did individually in the individual brainstorm condition. Consistent with
Carpenter (2007), we calculated the quantity of fraud risks as the number of risks listed that were
categorized as fraud risks by the participant, and we calculated the quality of fraud risks as those
risks that matched the frauds identified as actual frauds by the SEC for the case used in the
experimental materials. The overall unique number of ideas for the nominal groups is the sum of the
ideas listed by three individuals randomly formed into a group of three with redundancies deleted.
Part II of the experiment was restricted to 20 minutes, and all case materials were collected.
In Part III, all participants individually provided their final fraud risk assessments in either a
qualitative or quantitative form; thus, the response mode was consistent for each participant
throughout the experiment. Background information on the participants was also collected. Part III
took approximately ten minutes to complete.

RESULTS
To examine H1a and H1b, we analyze the data using an ANOVA with the number of fraud
risks (i.e., quantity) and the number of quality fraud risks as the dependent variables and with
brainstorm type and response mode as the independent variables. Table 1 contains the means of the
quantity and quality of fraud risks dependent variables, and Table 2 contains the ANOVA results of
these variables. H1a predicts that nominal groups (i.e., brainstorming individuals) will identify
more fraud risks (i.e., quantity) than brainstorming groups. However, H1b predicts that
brainstorming groups will identify more quality fraud risks than nominal groups of brainstorming
individuals. The quantity and quality of fraud risks are presented in Panels A and B of Table 1,
respectively. In Panel A of Table 2 the analysis shows that the mean number of fraud risks (i.e.,

7
The participants were asked to list each of these risks in addition to fraud risks to limit the attention drawn to fraud
risk assessments and because internal auditors are responsible for assessing these risks in practice. The results for
quantity of risks in each of these categories are similar to those found with quantity of fraud risks, our variable of
interest (i.e., the nominal groups identify more risks than the brainstorming groups).

Auditing: A Journal of Practice & Theory

August 2011
218 Carpenter, Reimers, and Fretwell

TABLE 1
Overall Quantity and Quality of Fraud Risksa,b
Panel A: Mean (Standard Deviation) Number of Fraud Risks (i.e., Quantity) by Treatment
for Individual Brainstorm Auditors (Nominal Group) and Brainstorming Audit Groups
Individual Individual
Group Brainstorm Brainstorm
Variables Brainstorm (Nominal Group) (Individuals)
Qualitative 5.60 6.50 4.28
(3.38) (4.28) (2.85)
Quantitative 4.80 8.33 4.89
(2.78) (4.97) (3.25)
Overall Mean 5.20 7.42 4.58

Panel B: Mean (Standard Deviation) Number of Quality Fraud Risks by Treatment for
Individual Brainstorm Auditors and Brainstorming Audit Groups
Individual Individual
Group Brainstorm Brainstorm
Variables Brainstorm (Nominal Group) (Individuals)
Qualitative 3.53 0.17 0.06
(1.41) (0.41) (0.24)
Quantitative 3.33 1.50 0.56
(1.54) (1.22) (0.62)
Overall Mean 3.43 0.83 0.31
a
Descriptive statistics for participants represent the cell mean for brainstorm type (individual brainstorm, group
brainstorm) and response mode (qualitative, quantitative) treatment combinations. n = 15 in each group brainstorm cell
(with 30 total groups), and n = 6 nominal groups in each individual brainstorm cell (with 12 nominal groups comprised
of a total of 36 randomly assigned individuals).
b
Participants were asked to brainstorm (either individually or as a group) listing all of the risks that are relevant to this
client. The quality of fraud risks was counted as the total number of risks designated as fraud risks by the participants
that matched one of the frauds actually identified by the SEC as a fraud that exists in the financial statements used in
our case materials.

quantity) for the nominal group is significantly greater (7.42) than that for brainstorm group (5.20),
(p = 0.039). This supports H1a. Additionally, in Panel B of Table 2 we show that the mean number
of quality fraud risks for the brainstorming group is significantly greater (3.43) than that for nominal
groups (0.83), (p , 0.001).8 This supports H1b.
Because internal auditors often work as individuals, the data allow us to analyze the
effectiveness of individual brainstorming. We performed additional ANOVA analyses comparing
the brainstorming groups to individuals alone. Brainstorming groups identified a mean 5.20 fraud
risks (i.e., quantity), and brainstorming individuals identified a mean 4.58 fraud risks, shown in
Panel A of Table 1. The analysis suggests that these are not significantly different (two tailed p =
0.420). However, brainstorming groups identified a mean 3.43 quality fraud risks, while
brainstorming individuals identified a mean 0.31 quality fraud risks, shown in Panel B of Table

8
Of the 30 brainstorming groups, two groups identified six quality fraud risks, six groups identified five, six groups
identified four, eight groups identified three, six groups identified two, one group identified one, and one group
identified zero quality fraud risks.

Auditing: A Journal of Practice & Theory

August 2011
Internal Auditors’ Fraud Judgments: The Benefits of Brainstorming in Groups 219

TABLE 2
ANOVA Results for Quantity and Quality of Fraud Risks
Panel A: Results of an ANOVA of Response Mode Between-Participants on Quantity of
Fraud Risks for Individual Brainstorm Auditors (Nominal Group) and Brainstorming Audit
Groups
Source of Variation df SS MS F-statistic p-value
Brainstorm Type 1 42.12 42.12 3.32 0.039*
Response Mode 1 2.29 2.29 0.18 0.674
Brainstorm Type 3 Response Mode 1 14.86 14.86 1.17 0.286
Error 38 482.83 12.71

Panel B: Results of an ANOVA of Response Mode Between-Participants on Quality of Fraud

Risks for Individual Brainstorm Auditors (Nominal Group) and Brainstorming Audit
Groups
Source of Variation df SS MS F-statistic p-value
Brainstorm Type 1 57.94 57.94 31.73 0.000*
Response Mode 1 2.75 2.75 1.51 0.227
Brainstorm Type 3 Response Mode 1 5.04 5.04 2.76 0.105
Error 38 69.40 1.83
* Denotes a one-sided p-value.

1. The analysis suggests that brainstorming groups provide significantly more quality fraud risks
than individuals (p , 0.001). Taken together, these analyses suggest that it is the interaction that
allows the brainstorming group to identify more quality fraud risks than brainstorming individuals,
as well as more than a nominal group, rather than brainstorming itself.
H2 predicts that group interaction will reduce the response mode bias. First, we confirmed that
participants’ quantitative risk assessments were significantly different from participants’ qualitative
assessments. The descriptive statistics, presented in Panel A of Table 3 and illustrated in Figure 1,
show that the mean initial qualitative fraud risk assessments of 4.14 is higher than the mean for the
quantitative treatment of 3.93; and the mean final fraud risk assessment for the qualitative treatment
of 4.22 is higher than that for the quantitative treatment of 4.13. The main effect for response mode
is marginally significant (one-tailed p = 0.059).9 Consistent with prior research, qualitative risk
assessments were higher than quantitative assessments (Reimers et al. 1993).
To test H2, we used repeated-measures ANOVA with brainstorm type (group brainstorm,
individual brainstorm, individual no brainstorm) and response mode (qualitative or quantitative) as

9
Consistent with prior research (e.g., Reimers et al. 1993), our response mode scales ranged from 0 to 6 to have
sufficient qualitative categories, rather than a larger scale of say 0 to 10 or 0 to 100. Therefore, while the
difference in means for the quantitative and qualitative conditions is marginally significantly different (one-tailed
p = 0.059), the difference in means is small. So while there is statistical significance, the economic significance
may be limited. However, we believe the economic significance has been shown as internal auditors in practice
are already implementing changes to their audit risk identification and assessment as a result of our study’s
findings. Our discussion with internal audit partners and directors (in firms and industry) suggests that because
these risk assessments serve as inputs into computerized audit risk models, even a small difference in risk
assessments can result in a large difference in the amount of testing to be performed, especially in the area of
fraud.

Auditing: A Journal of Practice & Theory

August 2011
220 Carpenter, Reimers, and Fretwell

TABLE 3
Fraud Risk Assessments
Panel A: Mean (Standard Deviation) Initial and Final Fraud Risk Assessments by Response
Mode for Group Brainstorm, Individual Brainstorm, and Individual No Brainstorm
Conditionsa
Group Individual Individual
Brainstorm Brainstorm No Brainstorm Overall Mean
Assessments Assessments Assessments Assessments
Variables Initial Final Initial Final Initial Final Initial Final
Qualitative 4.17 4.36 4.17 4.06 4.06 4.06 4.14 4.22
(0.83) (0.74) (0.62) (0.80) (0.64) (0.54) (0.74) (0.72)
n = 45 n = 45 n = 18 n = 18 n = 18 n = 18 n = 81 n = 81
Quantitative 4.02 4.40 3.94 3.89 3.67 3.69 3.93 4.13
(0.83) (1.03) (0.87) (0.83) (0.92) (1.03) (0.86) (1.03)
n = 45 n = 45 n = 18 n = 18 n = 18 n = 18 n = 81 n = 81
Overall Mean 4.09 4.38 4.06 3.97 3.86 3.88 4.03 4.18
(0.83) (0.89) (0.75) (0.81) (0.81) (0.83) (0.81) (0.89)
n = 90 n = 90 n = 36 n = 36 n = 36 n = 36 n = 162 n = 162

Panel B: Results of a Repeated-Measures ANOVA of Response Mode and Brainstorm Type

Between-Participants on Shift between Individual Auditor Initial and Final Fraud Risk
Assessmentsb
Source of Variation df SS MS F-statistic p-value
Between-Participants
Brainstorm Type 2 7.73 3.87 3.33 0.038
Response Mode 1 2.88 2.88 2.48 0.059*
Brainstorm Type 3 Response Mode 2 1.40 0.70 0.60 0.550
Error 156 181.12 1.16
Within-Participants
Shift 1 0.34 0.34 1.46 0.228
Shift 3 Brainstorm Type 2 2.11 1.05 4.49 0.013
Shift 3 Response Mode 1 0.14 0.14 0.59 0.443
Shift 3 Brainstorm Type 3 Response Mode 2 0.11 0.06 0.24 0.791
Error 156 36.59 0.24
* Denotes a one-sided p-value.
a
Descriptive statistics for participants’ fraud risk assessments; mean represents the cell mean for response mode
(quantitative, qualitative) and group type (group brainstorm, individual brainstorm, individual no brainstorm) treatment
combinations.
b
Participants were asked to provide a fraud risk assessment (the likelihood of financial statement fraud) on either a
quantitative seven-point Likert scale with endpoints labeled 0 (Extremely Unlikely) and 6 (Extremely Likely) or on a
qualitative scale seven-point scale with endpoints labeled 0 (Extremely Unlikely) and 6 (Extremely Likely).

between-participants variables, and shift (the initial risk assessment from Part I and final fraud risk
assessments from Part III) as a within-participants variable, allowing us to examine auditors’ final
fraud risk assessments while controlling for each individual’s initial judgment. H2 predicts the
difference in fraud risk assessments caused by response mode will be reduced by group interaction.
Thus, we expect an interaction between brainstorm type and the shift variables. We included a

Auditing: A Journal of Practice & Theory

August 2011
Internal Auditors’ Fraud Judgments: The Benefits of Brainstorming in Groups 221

FIGURE 1
Fraud Risk Assessments

3(Brainstorm Type) 3 2(Response Mode)

This figure illustrates the mean fraud risk assessments for the brainstorm type (group brainstorm, individual
brainstorm, individual no-brainstorm) and response mode (qualitative, quantitative) treatment combinations.
Participants were asked to provide a fraud risk assessment (the likelihood of financial statement fraud) on either
a quantitative seven-point Likert scale with endpoints labeled 0 (Extremely Unlikely) and 6 (Extremely Likely)
or on a qualitative seven-point scale with endpoints labeled 0 (Extremely Unlikely) and 6 (Extremely Likely).

control group of individuals who did not brainstorm or list risks (individual no-brainstorm).
Because brainstorming is our variable of interest, we need a control condition in which there is no
brainstorming to be sure that any results related to individuals are due to the act of brainstorming
(i.e., the generation of ideas) and not from the act of assessing fraud risk twice.
As predicted in H2, there is a significant shift 3 brainstorm type interaction (two-tailed p =
0.013). Thus, H2 is supported. Additional repeated measures ANOVA analysis shows that response
mode is marginally significant for individuals (one-tailed p = 0.059), but response mode is not
significant for the group brainstorm condition (one-tailed p = 0.378). Before the group
brainstorming, auditors’ qualitative risk assessments of 4.17 were significantly higher than the
auditors’ quantitative risk assessments of 4.02. However, after group brainstorming, the difference
between the two response mode conditions (4.36 and 4.40, respectively) is diminished for
interacting groups that brainstorm (p = 0.805).10 This provides additional support for H2.

SUMMARY AND CONCLUSION

In this research, we seek to determine if group brainstorming will improve internal auditors’
fraud judgments. Consistent with prior research, we find that nominal groups (i.e., a collection of

10
Additional analysis of the overall group fraud risk assessments from the 30 groups in the group brainstorm
condition also suggests that there is no significant difference ( p = 0.822) between the group fraud risk
assessments of those groups in the qualitative condition (4.27) and those in the quantitative condition (4.20).

Auditing: A Journal of Practice & Theory

August 2011
222 Carpenter, Reimers, and Fretwell

individual internal auditors who do not interact) who brainstorm individually identify more fraud
risks (i.e., quantity) than brainstorming groups. However, brainstorming groups identify more
quality fraud risks (actual frauds identified by the SEC) than nominal groups. By having individual
auditors brainstorm alone, we were able to separate the effects of brainstorming (i.e., generation of
ideas) and group interaction. Our results suggest that the benefit is derived by group interaction, not
the act of brainstorming (i.e., generation of ideas). These findings are important in this context as
internal auditors do not always work in groups.
Based on psychology theory on cognitive load, we predict and find that group interaction
reduces the response mode bias that has been found to be present in auditors’ risk assessments.
Specifically, auditors who assessed risk qualitatively provided higher fraud risk assessments than
did those who assessed risk quantitatively. However, after group brainstorming, there was no
difference due to response mode. This is another advantage of brainstorming in groups that has not
previously been identified.

REFERENCES
American Institute of Certified Public Accountants (AICPA). 2002. Consideration of Fraud in a Financial
Statement Audit. Statement on Auditing Standards No. 99. New York, NY: AICPA.
American Institute of Certified Public Accountants (AICPA). 2003. AICPA Practice Aid Series, Fraud
Detection in a GAAS Audit. SAS No. 99 Implementation Guide. New York, NY: AICPA.
Bamber, M., R. Watson, and M. C. Hill. 1996. The effects of group support system technology on audit
group decision making. Auditing: A Journal of Practice & Theory 15 (1): 122–134.
Carpenter, T. 2007. Audit team brainstorming, fraud risk identification, and fraud risk assessment:
Implications of SAS No. 99. The Accounting Review 82 (5): 1119–1140.
Church, B. K., and A. Schneider. 1995. Internal auditors’ memory for financial statement errors. Behavioral
Research In Accounting 17–36.
Dennis, A., and J. Valacich. 1993. Computer brainstorms: More heads are better than one. Journal of
Applied Psychology 78 (4): 531–537.
Diehl, M., and W. Stroebe. 1987. Productivity loss in brainstorming groups: Toward the solution of the
riddle. Journal of Personality and Social Psychology 53 (3): 497–509.
Dugosh, D., P. Paulus, E. Roland, and H. Yang. 2000. Cognitive stimulation in brainstorming. Journal of
Personality and Social Psychology 79 (5): 722–735.
Dusenbury, R., and J. Reimers. 2010. The Effect of Response Mode on the Recall of Information in a
Professional Judgment Task. Working paper, Florida State University.
Gallupe, R., L. Bastianutti, and W. Cooper. 1991. Unblocking brainstorms. Journal of Applied Psychology
76 (1): 137–142.
Gramling, A., M. Maletta, A. Schneider, and B. Church. 2004. The role of the internal audit function in
corporate governance: A synthesis of the extant internal auditing literature and directions for future
research. Journal of Accounting Literature (January): 194–217.
Hoffman, V., and M. F. Zimbelman. 2009. Do strategic reasoning and brainstorming help auditors change
their standard audit procedures in response to fraud risk? The Accounting Review 84 (3): 811–837.
Hunton, J., and A. Gold. 2010. A field experiment comparing the outcomes of three fraud brainstorming
procedures: Nominal group, round robin, and open discussion. The Accounting Review 85 (3): 911–
935.
Hurtt, K. 2010. Development of an instrument to measure professional skepticism. Auditing: A Journal of
Practice & Theory 29 (1): 149–171.
Institute of Internal Auditors (IIA). 1998. Standards for the Professional Practice of Internal Auditing.
Altamonte Springs: FL. Available at: http://www.theiia.org
Institute of Internal Auditors (IIA). 2003. Editorial Preface. In Research Opportunities in Internal Auditing,
edited by A. Bailey, A. Gramling, and S. Ramamoorti. Altamonte Springs, FL: The Institute of
Internal Auditors Research Foundation.

Auditing: A Journal of Practice & Theory

August 2011
Internal Auditors’ Fraud Judgments: The Benefits of Brainstorming in Groups 223

Institute of Internal Auditors (IIA). 2006. Organizational governance: Guidance for internal auditors.
Position paper. Available at: http://www.theiia.org
Institute of Internal Auditors (IIA). 2009. A world in economic crisis: Key themes for refocusing internal
audit strategy. White paper. Available at: http:/www.theiia.org/GAIN
Janis, I. L. 1972. Victims of Groupthink. Boston, MA: Houghton Mifflin.
Janis, I. L. 1982. Groupthink: Psychological Studies of Policy Decisions and Fiascoes. Boston, MA:
Houghton Mifflin.
Johnson, E. 1994. Auditor memory for audit evidence: Effects of group assistance, time delay, and memory
task. Auditing: A Journal of Practice & Theory 13 (1): 36–56.
Kinney, W. 2003. Auditing risk assessment and risk management process. In Research Opportunities in
Internal Auditing, edited by A. Bailey, A. Gramling, and S. Ramamoorn, 131–170. Altamonte
Springs, FL: The Institute of Internal Auditors Research Foundation.
Kirschner, F., F. Pass, and P. A. Kirschner. 2008. Individual versus group-based learning as a function of
task complexity: An exploration into the measurement of group cognitive load. In Beyond
Knowledge: The Legacy of Competence, edited by J. Zumbach, N. Swartz, T. Suefert, and L. Kester,
21–28. Amsterdam, The Netherlands: Springer.
Kirschner, F., F. Pass, and P. A. Kirschner. 2009. Individual and group-based learning from complex
cognitive tasks: Effects on retention and transfer efficiency. Computers in Human Behavior 25 (2):
306–314.
Levine, J., and R. Moreland. 1990. Progress in small group research. Annual Review of Psychology 41:
585–634.
Lynch, A., U. Murthy, and T. Engle. 2009. Fraud brainstorming using computer-mediated communication:
The effects of brainstorming technique and facilitation. The Accounting Review 84 (4): 1209–1232.
Mullen, B., C. Johnson, and E. Salas. 1991. Productivity loss in brainstorming groups: A meta-analytic
integration. Basic and Applied Social Psychology 12 (1): 3–23.
Nunamaker, J., A. Dennis, J. Valacich, D. Vogel, and J. George. 1991. Electronic meeting systems to
support group work. Communications of the ACM 34 (7): 40–61.
Osborn, A. 1957. Applied Imagination. Rev. edition. New York, NY: Scribner.
Petty, R., S. Harkins, and K. Williams. 1980. The effects of group diffusion of cognitive effort on
attitudes: An information processing view. Journal of Personality and Social Psychology 38 (1):
81–92.
PricewaterhouseCoopers LLP. 2009. Business Upheaval: Internal Audit Weighs Its Role Amid the
Recession and Evolving Enterprise Risks. New York, NY: PwC.
Reckers, P., and J. Schultz. 1993. The effects of fraud signals, evidence order, and group-assisted counsel
on independent auditor judgment. Behavioral Research In Accounting 5: 125–144.
Reimers, J., S. Wheeler, and R. Dusenbury. 1993. The effect of response mode on auditors’ control risk
assessments. Auditing: A Journal of Practice & Theory Fall: 62–75.
Sniezek, J., and R. Henry. 1989. Accuracy and confidence in group judgment. Organizational Behavior and
Human Decision Processes 43: 1–28.
Trotman, K. 1985. The review process and the accuracy of auditor judgments. Journal of Accounting
Research 23 (2): 740–752.
Trotman, K., R. Simnett, and A. Khalifa. 2009. Impact of the type of audit team discussions on auditors’
generation of material frauds. Contemporary Accounting Research 26 (4): 1115–1142.
Trotman, K., and P. Yetton. 1985. The effect of the review process on auditor judgments. Journal of
Accounting Research 23 (Spring): 256–267.

Auditing: A Journal of Practice & Theory

August 2011
224 Carpenter, Reimers, and Fretwell

APPENDIX A
RESPONSE MODE SCALES

The response mode independent variable was varied between participants in all parts of the
experiment. Participants were randomly assigned to the qualitative or quantitative condition and
were asked to provide risk assessments on the following scales:

Quantitative Scale

Likelihood of Fraud: Financial statement fraud (an intentional act that results in a material
misstatement in the financial statements) is present.

Qualitative Scale

Likelihood of Fraud: Financial statement fraud (an intentional act that results in a material
misstatement in the financial statements) is present.

Auditing: A Journal of Practice & Theory

August 2011
Copyright of Auditing is the property of American Accounting Association and its content may not be copied or
emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.