Doing Quantitative Field Research in Management Accounting

By Shannon W. Anderson and Sally K. Widener

Jesse H. Jones Graduate School of Management, Rice University, USA
Department of Accounting and Business Information Systems, University of Melbourne, Australia
Abstract: This paper provides practical guidance to management accounting researchers on the design and
execution of field studies that use quantitative data analysis to test or build theory. the purposes of this paper for
conducting field research and provide a brief overview of the accomplishments and failings of recent field
research in management accounting. We then turn to the doing of quantitative field research, discussing
practical considerations related to the role of theory; site selection; and data identification, collection and
preparation. Finally, it reflect on how field research practices may be amended to address some of the criticisms
of prior field research
1. Introduction
Management accounting is the process of identifying, measuring, accumulating, analyzing, preparing,
interpreting, and communicating information that helps managers fulfill organizational objectives (Horngren et
al., 2002: G6). Management accounting takes place in a specific organization at a specific point in time to meet
unique needs for management control and decision support so there are few rules for how management
accounting must be done. The objective of this paper is to provide practical guidance to management accounting
researchers about important considerations in the design and execution of field studies that use quantitative data
analysis to test or build theory. Accounting researchers formal training rarely extends to the field (Bennis &
OToole,2005; Shields, 1997; Young, 1999). This paper contributes to the literature on field research methods a
discussion of issues that confront the management accounting researcher who aims to use quantitative analysis
of data obtained in the field for rich description,for theory building, or for theory testing.
2. Quantitative Field Research: Meaning and Purpose
2.1. What Do Author Mean by Field Research?
There is the distinction that has been made in the management literature between case and field research
contains an implicit assumption that the firm is the appropriate unit of analysis. Field research is portrayed as
improving upon case study research because of the opportunity to consider firm-level variation and to generalize
results (albeit in a limited statistical sense) to more than one firm. However, management accounting research
covers a broadarray of questions for which different units of analysis are relevant (e.g., individuals, work teams,
organizational sub-units). Thus, we conclude that it is arbitrary for management accounting researchers to define
the unit of analysis for field research in the abstract without consideration of the research question.
Understandably, if the unit of analysis is the firm and field research costs escalate with the number of firms
studied, most field research will tend toward theory building rather than theory
testing. This is consistent with Yins (2003) observations and with Ferreira & Merchants (1992) characterization
of published field research. However, when a lower level of analysis (e.g., individual or work team) is warranted, it
is quite possible that theory testing may be accomplished in the field using fewer firms and at reasonable cost. 5
Whether a theory can be tested in a field setting with only a few organizations depends on the unit of analysis of
the theory and whether higher order effects (e.g., firm, industry, or economy-wide effects) are correlated with the
lower-level effects of interest. The definition of field research to studies that rely solely or even primarily on
interviews and observations. Author do not rule out the use of archival data in field research if an understanding
of the data is derived from direct and indepth contact with members of the firm. Author accept the importance of
sustained interactions with organizational members in the natural field setting and we admit the possibility that
researchers may need to adapt in response to these interactions, but we reject restrictions on the objective of the
study (e.g., theory testing versus theory building), the mode of data collection, and the number of firms studied.
2.2. What Do Author Mean by Quantitative Field Research?
Quantitative field research in relation to the nature of the analysis to which the data are subjected. Specifically,
Author focus on field research that uses data that may be represented numerically and are of a quantity and
quality to support empirical analysis using parametric or non-parametric statistical methods. The terms of the
number of firms studied Depending on the research question, this may imply the need for multiple time periods,
multiple individuals, multiple teams or other organizational units, or multiple firms. empirical analysis of numeric
data used in theory testing do not constrain the definition of quantitative field research in this manner.
Quantitative field studies removes from consideration studies that use numeric data primarily to corroborate or
extend interview data. three sources of numeric data as follows: (1) measured data, which are authentic numeric
data in the native state in which it is captured by the organization or individual (e.g., company records used in a
field study by Banker et al., 2000); (2) latent data that are derived from measured data (e.g., as in Ittner et
al.,1997); and (3) latent data that are measured through researcher intervention (e.g., coded interview data as
employed by Abernethy & Lillis, 1995; survey data as employed by Epstein & Widener, 2005; and coded
observations as employed by Anderson et al., 2002).
2.3. The Use of Quantitative Data Analysin to Achieve Different Purposes
A common progression in building systematic knowledge involves first careful observation and description,
followed by theory development, and then testing the theory in different settings to determine its relevant domain,
Field research that tests theory often employs quantitative analysis to determine whether central tendencies in
numeric data are broadly consistent or inconsistent with theoretical predictions.
Atkinson & Shaffir (1998: 63) propose that field studies that test theory should meet four conditions:

a.
b.

The conditions of the test should be consistent with the underlying assumptions or axioms of the theory.
The test should define clearly and with good reason the test results that support the theory and the test
results that would contradict the theory.
c. The test should be unbiased in the sense of providing a reasonable probability that it could uncover
evidence that could confirm or contradict the theory.
d. The test should define and measure accurately the artifacts for the theorys variables.
3. Twenty Years of Management Accounting Field Research: Accomplishments and Shortcomings
Papers that were published in the 1980s are widely associated with contemporary field research in management
accounting : Hopwood (1983: 302) argued that accounting had divorced itself from the organization and called for
more research on the design and use of accounting systems within the context of an organization Kaplan (1983,
1984) argued that innovative cost accounting and management control practices were invisible to traditional
academic research, which tended to study central tendencies. in large populations Ferreira & Merchant (1992)
evaluate the quality of field research in management accounting by comparing the 82 studies to a set of fieldwork
standards (Bruns & Kaplan, 1987) and categorize the papers according to their subject matter, research design,
data presentation and interpretation, and practical implications They conclude that published field studies in
management accounting have explored interesting questions, usually in conjunction with emerging and
innovative practices, and that most of the studies have progressed from description to testing or developing
theory. Merchant & Van der Stede (2005) conduct a survey of revisions to leading accounting textbooks during
the same period. They trace the five major areas of innovation in management accounting practice to research
that had its genesis in the field and conclude that field research has been exceptionally valuable at producing
usable knowledge as compared to other research methods during the last 20 years. In spite of these
contributions, field research is not without weaknesses. Ferreira & Merchant (1992) attribute problems with field
research to poor research design, poor data presentation, and flawed data interpretation. Ferreira and Merchant
also criticize field studies as being inadequately connected to prior research literatures. For example, in failing to
select sites that differ in controlled ways from previous studies, each field study can easily become a one off
study that does not contribute to a stream of research and cumulative knowledge. Using a smaller set of primarily
North American journals, Shields (1997) finds that the field research method has a low frequency of use; 10 out
of 152 articles published during the 19901996 period used field study methods. He posits that the lack of
published field research stems from inadequate skills in conducting field research, a dearth of colleagues from
whom to obtain feedback and with whom one might collaborate, long research cycle times that are in conflict with
short-tenure clocks, limited access to field sites, and editorial biases against field research. Bennis & OToole
(2005:99) trace broader failings in management education to failings of the research community. They criticize
the form of scientific method commonly employed by management scholars for involving minimal time in the field
discovering the actual problems facing managers and conclude that,....because they are at arms length from
actual practice, they often fail to reflect the way business works in real life. Unfortunately, Bennis and OToole
are less optimistic than Young or Shields that extant academic institutions have the means or motivation to
support scholarship that is more connected to the practice of management.
4. Key Decisions, Choices, and Contributions of Quantitative Field Research
Table 1 categorizes the field research papers the author use to illustrate their arguments according to the nature
of the data analysis and the papers research purpose.

4.1. Role of theory

The characterization of theory as being unformed or changing dynamically tends to be applied in
descriptive and theory-building research. Eisenhardt (1989: 536) illustrates an extreme view in her
discussion of how to build theory in the field when she states theory-building research is begun as
close as possible to the ideal of no theory under consideration and no hypotheses to test. Other
authors describe a dynamic relation between field research and extant theory, ranging from advising
researchers to have no ex ante theoretical expectations to suggesting that researchers be informed
about competing theories, but open-minded to discovery of new theory (Baxter & Chua, 1998;
Shields, 1997). While Ferreira & Merchant (1992) define field research in terms of whether the
research design adapts as researchers gain more knowledge from the field, they also criticize field
studies as being inadequately linked to the research literature and failing to contribute to a cumulative
body of management accounting knowledge. Thus it is clear that even if theory-building field research
is not conditioned by theory, there must be some ex ante rationale for expecting grounded theory
development in the selected field setting to add to our understanding of what is broadly defined as
management accounting. If the field studys goal is theory testing, then the success of the project
depends critically on the linkage between the theory and the test. Theory serves to define the
appropriate research setting and unit of analysis, to identify key variables and key informants, and to
specify the form of the empirical test (e.g., model specification, identification of parameters of interest,
identification of appropriate statistical analysis
tools). Related to this point, theory serves to define databoth qualitative and quantitativethat are
necessary to permit the researcher to rule out competing theories or to disconfirm theory. Thus, in
quantitative field research, theory precedes and conditions all aspects of the field project. Although
the theory that the authors sought to test (on the moderating effect of information technology on costs
of product complexity) was unchanged, the emergent understanding of the data generating process

was critical to the quality of the data and proper specification of the tests (Atkinson & Shaffir, 1998).
One condition for successful theory testing is the fidelity with which the measured variables represent
the variables of the theory. In how many studies does the theory and literature review seem utterly
disconnected from the empirical analysis because the variable measures do not correspond to the
theory? In the end, even if the researcher finds significant results, the reader does not believe that
the theory has been tested.

Author have found it equally challenging in quantitative field Studies as when labels that companies
assign to measured data do not correspond to academic definitions or when survey questions
inadvertently use language that has special meaning inside a company that differs from general use.
Author have found that it typically takes several conversations as well as extensive exploratory data
analysis for the researcher to be able to relate a companys measure of capacity to economic notions
of, for example, theoretical capacity, normal capacity, actual capacity, and peak load capacity. Young
& Selto (1993) is an example of researchers conducting an open-minded search for data on
performance. Their objective was to identify and measure performance improvements associated
with implementation of just-in-time manufacturing methods. During the first 6-months in the field, they
identified six different archival measures of performance appropriate for their site: cycle time
efficiency, yield rate, defective rate, production schedule adherence, product cost efficiency, and
number of manufacturing process problems. In sum, theory is emergent in field studies aimed at
grounded theory development and is a strong preconditioning factor for field research aimed at theory
testing. However, even theory testing requires dynamic adaptation of the researcher. Once the theory
is identified, quantitative field studies aimed at theory testing must grapple with pragmatic issues of
research design, data collection, and analysis in an effort to ensure that the tests are aligned with the
theory.
4.2. Site Selection
Selecting a site suitable for theory testing occurs simultaneously with the practicalities of ensuring that
data exist to allow the proposed hypotheses to be tested. A sequence of (1) identifying
(opportunistically) a site where researchers can obtain access, (2) searching for data, and (3)
selecting a theory to test is generally not recommended. Practicalities are such that research only
rarely progresses linearly from theory to identifying the properties of an ideal site, to negotiating
access to that site, and to discovering data that meet all of the researchers needs.
In some field research, it may be possible to unearth latent data through interviews, direct
observations, and surveys. However, if the researcher is interested in using archival data that are

measured and captured by the firm, then site selection requires a simultaneous evaluation of the
availability of measured data. Chenhall (1997) and Davila (2005) are excellent examples of selecting
a group of firms that are in equilibrium with respect to a certain feature of interest. For Chenhall
(1997), it was important to obtain an adequate sample of firms that practiced total quality
management techniques and used manufacturing performance measures to evaluate managerial
performance. This example illustrates an approach to identifying multiple field research sites that
resemble an ideal profile that is derived from the theory to be tested. Another example of a field
study that selects a large number of firms that are distinct with respect to certain characteristics is
Davila (2005). To generate a large enough sample of firms (95) to enable empirical testing Davila
capitalized on the disproportionate representation of small, young, technology-oriented firms in
Californias Silicon Valley. In Anderson & Lanen (2002), the authors became involved with the firm
when the companys top manufacturing manager sought to understand the impact of product mix
complexity on manufacturing performance. The company was committed to an involved relationship
with educational and research programs of the university, and had identified product complexity as a
topic of interest. However, in spite of the aligned interests of managers and researchers, extensive
plant visits and interviews at six production facilities revealed that the data generated in the factories
were too aggregate and imprecise to support empirical analysis that would extend the literature on
complexity costs. The first example is of a linear search for a firm that permits testing of a rather
specific theory. The success of this approach owes much to the extensive network of practicing
managers to whom senior faculty mentors had access as well as to the willingness of these
colleagues to help the researcher find and gain access to an exemplar organization that permitted
powerful tests of the theory. Thus, we concur with Youngs (1999) counsel that young aspiring field
researchers ally themselves with senior faculty and administrators who know and are involved with
practicing managers. During the research design stage, the researcher must determine the proper
unit of analysis for the theory being tested, and then assess the probability that they will be able to
detect a small effect. If the theory is at the level of the firm, the researcher should consider whether
they can create a large enough sample to rule out a possible explanation of no findings due to low
statistical power. In sum, the researcher pursuing a quantitative field research study must consider
various factors when selecting an appropriate site including the availability of data, the
appropriateness of the company for the study, the appropriate unit of analysis, and whether adequate
statistical power is likely to be obtained for testing the theory. For quantitative field studies that have
theory testing as a goal, careful site selection is critical to meeting the first and third conditions that
Atkinson & Shaffir (1998) require of successful theory testing in the field: that the conditions of the test
(i.e., the setting, the unit of analysis) are consistent with the axioms of the theory, and that the test
provides reasonable probability (i.e., power) to confirm or contradict the theory.
4.3. Data Identification, Collection, and Preparation
4.3.1. Overview of Data
The natural state of the data that are needed for the research question will determine what data
collection tools are necessary as well as the most likely set of analysis methods to employ. However,
this overstates the case, because at times it is possible to identify good proxies for latent data in
measured data, and similarly, there are opportunities to create latent data from measured data.
Researchers also often substitute latent data that emerge from measurement interventions for
measured data and for latent data that are created from measured data, as for example, when survey
respondents are asked to report otherwise unverifiable performance or are asked to opine on matters
that are measured or measurable using pre-existing data. This approach to data collection has proven
the most difficult in publishing field-based research. As might be expected, it opens the researcher to
questions about the likely effects of measurement intervention and is often associated with concerns
about endogeneity and circularity that are not easily addressed.The most important question is not
how to obtain a quantitative measure for each construct that is demanded by the research question,
but how to obtain many measures for each construct (which may include multiple respondents to an
interview or survey) and how to obtain measures that have different natural states (which requires
supplementing interviews and surveys with measured data or latent data derived from measured
data). Together these approaches increase the likelihood of quantitative field research meeting
Atkinson and Shaffirs fourth condition for theory testing: that variable measures represent with high
fidelity the constructs that are critical to the test of the theory.
4.3.2. Collecting Measured Data in the Field Setting
Although the researcher is not involved in the construction of measured data, the researcher still has
an important role to play in assembling an appropriate database and in preparing the data for use.
Even when the data are archival accounting records, the researcher must typically make important
selections from a large database of potential measures. Although the temptation is to take

everything, this only delays the careful culling of important measures. Atkinson & Shaffir (1998: 44)
quoting Gusfield (1995) say: Data that do not stay close to the events, actions, or texts being studied
are always suspectyWhat Geertz called thick description is the ideal, not always achieved but always
to be aimed at. Furthermore, if we are trying to understand management accounting in its natural
context, then we need to understand the key actors interpretation and perception of the situation. A
somewhat unique opportunity when measured data are employed is the possibility of longitudinal
analysis.25 Ittner et al. (1997), in their study of activity- based costing, provide an illustration of a
longitudinal study that uses measured data. Using timeseries data on monthly cost, revenue, and
operations, the authors generated insights about the relation between the cost hierarchy
classifications and costs. These relations are often hard to detect in cross-sectional studies where
variances are averaged out of the sample. Longitudinal analysis brings a different challenge namely,
ensuring that the data have been collected and used consistently over time. It is important for the
researcher to determine whether what is measured has changed over time (e.g., as when accounting
classifications change or boundaries within the organization change). the survey questions were
consistent across time, sometimes the scale changed (e.g., a change from a 5-point to a 7-point
scale, or a change of endpoint anchors for the scale). In addition to the content and structure of the
data, it is also important to consider how the use and visibility of the data has changed over time.
Relatedly, perhaps the measure has had consistent visibility; however, during a portion of the time
period it was linked directly to compensation. Archival research that is not associated with sustained
direct contact with the data generating process is unlikely to uncover these issues that can seriously
compromise empirical tests.
4.3.3. Collecting Survey Data in the Field Setting
If a survey is used to collect data in the field, researchers must consider the timing of the survey in
relation to the timing of other fieldwork. For example, if the researcher combines interviews with a
survey, which should come first? Should the survey precede the more personal interaction with
informants, or follow it? Studies that are predominantly survey-based, may prefer the survey to be
informed by the interviews. The interviews set the stage and define terminology so that survey
respondents are ready to efficiently provide responses (Ferreira & Merchant, 1992). Often there is
an iterative process with data collection alternating between interviews and various stages of
development, refinement, and use of the survey instrument. Using knowledge gained during their
preliminary visits, the researchers adapted the survey to fit a modern technology firm. This was
followed by a pre-test on several groups of plant workers and managers to ensure that the language
was meaningful for their particular research site The survey was revised and administered, at which
time the researchers realized that many of the survey respondents were not fluent in English. In
hindsight, Young and Selto acknowledge that they would have used simpler language if this
had been known. One might use the survey to collect the more easily measured and defined items,
leaving the interview to cover more complex concepts. Another consideration in sequencing different
modes of data collection from a single informant is that the sequence may affect responses and
introduce unwanted bias. By doing a survey first, the interview can be done more efficiently since the
respondent will already understand the researchers interests. The interview can take the place of
open-ended questions on the survey. If one assumes that surveys suffer less from respondent
reactivity, the sequence of survey first, interview second may be less prone to the researcher
affecting all of the data in undesirable ways. However, if the researcher includes complex concepts in
the survey, it becomes more compelling to use the interview first as a means of building shared
understanding of key concepts. Then the survey can be used to elicit respondent opinions related to
the concepts with some assurance that measurement error has been mitigated. In this manner the
strength of having researcher involvement to explain complex issues is harnessed, but at the cost that
undesirable researcher effects could influence both the survey and the interview. Surveys can also be
used to supplement archival data; thus, researchers need to consider opportunities to establish
construct validity with survey data and measured data. Since the survey data typically are drawn from
multiple observers while the archival data may present a single measure that applies to the entire
organization, researchers must decide how many survey respondents are needed and who will be
adequately informed about the phenomena in question. Finally, when data are collected using surveys
in the field the researcher must consider the distribution mechanism. Survey administration is always
fraught with concerns about response bias and response rate, and there are methods for mitigating
these concerns for mail surveys (e.g., Dillman, 1978). Alternatively, the researcher may decide to
administer the survey in person. This has several desirable properties since researchers can mitigate
one common criticism of surveysthat an unknown and/or unwanted person completed the survey. In
addition, if respondents are allowed time during their work hours to complete the survey they may
infer that the survey is more important than they might otherwise.

4.3.4. Collecting Interview Data in the Field Setting

Certainly measurement error exists in latent variables, especially when constructed from qualitative
data. The primary concern for researchers using this technique is to reduce measurement error and
reduce the need for readers to place trust in the researcher. A better alternative is to involve a third
person to recode the data after the coding scheme has been agreed upon. This persons coding can
then be used to establish inter-rater reliability. when researchers use interview data to create numeric
data they provide the readers with a roadmap of the coding schema. This will help reduce the
absolute trust that readers must place in researchers. Due to a lack of suitable and available
constructs to measure both flexibility and integrative liaison devices, the authors chose to use a
semistructured interview approach and code the interview data into constructs. They produced interrater reliability metrics since each interview protocol was coded by both researchers. In addition, they
included the coding schema in the paper for the readers. The researchers turned the semi-structured
interview data into Likert-scale questions ranging from 1 to 5, which were subsequently used in
statistical analyses. Another important part of measurement error is that the coding of qualitative data
is always subject to bias introduced by the researcher both during the interview and when coding the
data (Lillis, 1999). Huber & Power (1985) identifies four sources of bias (or in accurate information) in
interview data: (1) the respondents have a motivation to provide biased information, (2) result of
perception and/or cognitive limitations on the part of the respondent, (3) informant does not possess
adequate information, and (4) the researcher is not eliciting information properly. The use of a semistructured questionnaire is necessary, especially when multiple researchers might interview different
respondents. For data that will be used in quantitative analysis, it is important that the same question
be asked, in the same manner, to all respondents to ensure that the raw response data is available to
code. It is important that the right person in the organization be selected as an informant (Huber &
Power, 1985). Researchers must ensure they not only understand the coding scheme, but understand
how it will translate into a quantifiable variable, and also ensure that the measured variable will reflect
the theoretical definition of the variable. The use of multiple coders to both establish reliability and
review the interview transcripts for inconsistencies is an absolute necessity and can also help reduce
bias (Abernethy & Lillis, 1995). The interview method is a powerful technique for gathering latent data
that are amenable to quantitative analysis. Unfortunately, it does not appear that many researchers
exploit this type of data although it can be very useful to establish construct validity, to triangulate
findings, and to measure variables that are not well-defined.
4.3.5. Summary of Data Identification, Collection, and Preparation
Jick (1979) presents an interesting and informative discussion of various forms of triangulation and
how to mix qualitative and quantitative data. He (1979: 602) states, There is a distinct tradition in the
literature on social science research methods that advocates the use of multiple methods.
Quantitative researchers are urged to draw on qualitative observations to validate and interpret
results, and clarify unexpected findings (Jick, 1979). Instead of using multiple methods simply to
round out the picture, researchers should attempt to obtain measures for each construct from more
than one source.
5. Conclusion and Personal Observations
For field research that aims to test theory, the theory must be well-defined before powerful empirical
tests can be constructed. Another early and critical decision that the researcher must make is the
selection of the field site. In order to undertake a successful quantitative field study it is imperative that
the researcher evaluate data availability and choose sites and informants that provide a powerful test
of the theory. T he importance of proper measurement in the field and the advantages a field site can
offer in terms of combining data from multiple methods in triangulating
variable measures. The academy benefits when many research traditions thrive and when
researchers gravitate to questions and methods that suit their skills and talents. The researcher must
consider the strengths and weakness of alternative research methods. Only if field research offers
compelling advantages over other methods, if there is a reasonable expectation of gaining access to
organizations or individuals that will provide data for a powerful test of the theory, and if the
researcher is passionate about working in organizations and is willing to bear the uncertainty
associated with field research and its publication should it be pursued. Regrettably, perceived
uncertainty is often the deciding factor that biases against protracted involvement with firms
(Bennis & OToole, 2005). However, for the researcher who is committed to understanding
management accounting in its natural setting, an obvious response is that other paths are far more
risky, if by risk we consider both the likelihood and the magnitude of innovation associated with
plowing the well-turned field of public accounting data versus collecting private data from a promising
site that is chosen to enhance the power of the test or the likelihood of revealing new theoretical
relationships.