Potter Methods of Foreign Policy Analysis

Methods of Foreign Policy Analysis
Oxford Research Encyclopedia of International

Studies
Philip B.K. Potter
Print Publication Date: Mar 2010 Subject: Foreign Policy Online Publication Date: Nov 2017
DOI: 10.1093/acrefore/9780190846626.013.34
Summary and Keywords
Foreign policy analysis (FPA) is the study of how states, or the individuals that lead them,
make foreign policy, execute foreign policy, and react to the foreign policies of other
states. This topical breadth results in a subfield that encompasses a variety of questions
and levels of analysis, and a correspondingly diverse set of methodological approaches.
There are four methods which have become central in foreign policy analysis: archival
research, content analysis, interviews, and focus groups. The first major phase of FPA
research is termed “comparative foreign policy.” Proponents of comparative foreign
policy sought to achieve comprehensive theories of foreign policy behavior through
quantitative analysis of “events” data. An important strand of this behavioral work
addressed the relationship between trade dependence and foreign policy compliance. On
the other hand, second-generation FPA methodology largely abandoned universalized
theory-building in favor of historical methods and qualitative analysis. Second-generation
FPA researchers place particular emphasis on developing case study methodologies
driven by social science principles. Meanwhile, the third-generation of FPA scholarship
combines innovative quantitative and qualitative methods. Several methods of foreign
policy analysis used by third-generation FPA researchers include computer assisted
coding, experiments, simulation, surveys, network analysis, and prediction markets.
Ultimately, additional attention should be given to determining the degree to which
current methods of foreign policy analysis allow predictive or prescriptive conclusions.
FPA scholars should also focus more in reengaging foreign policy analysis with the core of
international relations research.
Keywords: foreign policy analysis, methodological approaches, comparative foreign policy, events data analysis,
case study methodologies, network analysis, prediction markets, foreign policy behavior
Page 1 of 29
PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

(c) Oxford University Press USA, 2019. All Rights Reserved. Personal use only; commercial use is strictly prohibited (for
details see Privacy Policy and Legal Notice).
Subscriber: Universidad de Chile; date: 06 March 2019

Introduction
The periodic reassessment of research methods is important to the vitality of any
academic discipline, but it has particular salience for a relatively young field such as
foreign policy analysis (FPA). Hudson and Vore (1995:221) acknowledge as much in their
review of the FPA literature, noting that, “in the study of foreign policy decision-making,
the issues are not theoretical but methodological.” I define foreign policy analysis as the
study of how states, or the individuals that lead them, make foreign policy, execute
foreign policy, and react to the foreign policies of other states. This topical breadth
results in a subfield that encompasses a variety of questions and levels of analysis, and a
correspondingly diverse set of methodological approaches. This essay surveys FPA’s
methodological development from its inception to the present and, in the process,
outlines the body of existing methodological practice and identifies opportunities for
future progress. The objective is to provide both an indication of the role that various
quantitative and qualitative methods play in the FPA literature and an entryway for
contemporary researchers seeking to apply these approaches to future work. Where
appropriate, the reader is directed to more specific guides to the intricacies and
execution of each method.
For the sake of organizational clarity, this review follows a stylized format roughly based
on Neack, Hey, and Haney’s (1995) concept of “generational change” in foreign policy
analysis. The section that immediately follows is partially archeological, that is, it surveys
methods of events data analysis that were important to the early development of FPA, but
in some cases have fallen out of widespread usage. The second section, which surveys
qualitative methods, most closely reflects the current state of the art in the discipline.
The third and final section addresses both cutting-edge and underutilized approaches.
The Methodological Origins of Foreign Policy

Analysis
The unique historical context and intellectual environment of the early 1950s –
specifically, the Cold War and the behavioral revolution – crucially shaped the early
methodological development of foreign policy analysis. These origins have proven central
to the methodological arc of the sub-discipline.
FPA was born of the opportunities presented by the largely atheoretical nature of
historically oriented diplomatic analysis and the exclusion of political leadership and
decision-making from the prevailing theories espoused by mainline international
relations. Prior to the advent of FPA as a distinct subfield, the study of foreign policy
relied on traditional methods and had long been the domain of political historians and
diplomatic strategists in the tradition of thinkers such as Thucydides and Machiavelli.
Early FPA researchers saw this longstanding tradition as part of their heritage, but,
inspired by the methodological imperatives of the behavioral revolution, believed that
systematizing the study of foreign policy would lead to progress in the form of
Page 2 of 29


generalizable and cumulative findings. Thus, from its inception, FPA was an explicitly
theoretical exercise aimed at uncovering the systematic elements of foreign policy
interactions, and the methods deployed reflected this.
Simultaneously, in response to the near monopoly of system-level theory in international

relations, the pioneers of FPA argued that individual leaders or groups of decision makers
are often the primary drivers of outcomes in international interactions (Snyder et al.
1954). Thus, at the very core of FPA’s intellectual identity lies a revisionist methodology
(vis-à-vis diplomatic history) applied to a revisionist conception of the basic unit of
analysis (vis-à-vis mainline international relations).
The strategic environment, specifically the position of the US in the early Cold War, also
figured prominently in the early development of FPA methods. In the face of this
protracted geopolitical conflict, American political leaders became unusually involved in
the FPA academic endeavor. The promise of concrete conclusions and general enthusiasm
for “scientific” approaches to political problems that stemmed from the success of the
Manhattan Project led the US government to invest large sums in early FPA efforts. With
funding came the expansion of major research centers such as the Rand Corporation and
the Brookings Institution that were instrumental to the maturation of FPA as a subfield
and methodological approach in international relations. However, the money and
attention from the policy community came with strings attached – most notably, an
expectation for immediately relevant research. Over time this requirement became
increasingly difficult to reconcile with the relatively high uncertainty surrounding
quantitative estimates of foreign policy phenomenon.
The first major phase of FPA research that emerged from this crucible is termed
“comparative foreign policy.” Proponents of comparative foreign policy argued that
controlled comparison of the domestic sources of external conduct across different
countries could produce comprehensive theories of foreign policy behavior.
Methodologically speaking, these scholars sought to achieve these ends primarily
through quantitative analysis of “events” data, which I describe in detail in the section
that follows. However, this transition to quantitative analysis was, at least in part, a
refinement of even earlier attempts to develop a more robust understanding of the
foreign-policy decision making process. Snyder, Bruck, and Sapin’s (1954) classic essay
was arguably the first to encourage international relations scholars to reopen the “black
box” of the state in order to study the actions of individual leaders. A significant body of
early qualitative case study research flowed from this call to arms. To take just two
examples, Paige (1968) took a decision making approach to understanding the origins of
the Korean War, while Allison (1971) followed along similar lines with his well-known
study of the Cuban Missile Crisis.
The decision making school provided a useful groundwork, particularly by identifying the
leader as a crucial unit of analysis, a tradition that has persisted in FPA ever since.
However, the developments of the behavioral revolution eventually overtook the primarily
qualitative methods of these early FPA scholars. An increasing premium was placed on
Page 3 of 29


the generalizability garnered by operationalizing foreign policy interactions numerically

and analyzing them quantitatively. This transition gave rise to the comparative foreign
policy literature, which maintained an emphasis on decision making and scientific
analysis, but moved away from case study analysis in favor of events data.
Comparative Foreign Policy and Events Data Analysis
The demand for foreign policy research that was scientific, generalizable, and policy
relevant caught nascent foreign policy analysts unprepared. Where other areas of
political science could respond to the challenge presented by the behavioral revolution
with numerical data already at their disposal, the traditional fodder for diplomatic
analysis – histories, documents, interviews, biographies, and memoirs – were less easily
reduced into the sort of data necessary for rigorous, quantitative hypothesis testing.
This reality set foreign policy analysis somewhat behind other areas of political science
because it had to overcome two distinct obstacles. First, new data had to be collected
that was better suited to statistical analyses. Second, methods had to be developed with
which to analyze these data within a behavioral framework. Among others, Rosenau
(1966; 1968), McClelland (1970), and Brecher et al. (1969) took up these early
challenges.
These early foreign policy analysts sought to develop a quantifiable unit of foreign policy
interaction. McClelland conceived of this core unit of analysis as the foreign policy
“event,” which is simply a formalized observation of a conflictual or cooperative
interaction between states. McClelland’s intention was to fill the gap between the
traditional narrative approach to foreign policy analysis and empirical techniques that
relied upon discrete quantifiable data that could be explored in statistical analyses
(Schrodt 1994). In effect, the foreign policy event takes a qualitative observation of
foreign policy interaction and reduces it to a numerical or categorical form suited for
statistical analysis.
The process of generating events data was and is time-consuming and costly. It is most
commonly accomplished through the content analysis of thousands of newspaper reports
on the interactions among nations in light of a previously defined set of criteria or
codebook. Each observation uncovered in this way is then assigned some numerical score
or a categorical code, which can then be analyzed quantitatively (Schrodt 1994). This
potentially lengthy process requires that the researcher accomplish some or all of the
following: identify sources, identify a period of analysis, create or borrow a coding
scheme, train coders, generate the data, and check for reliability.
Foreign policy scholars have generated a significant number of important events datasets
that remain central to quantitative methods of foreign policy analysis. The best of them
are impressive collections offering decades-long periods of analysis, coverage of many
countries (if not the entire international system) and standards of intercoder reliability
well in excess of 80 percent (Burgess and Lawton 1972). The paragraphs that follow
describe a subset of the available data. Particular attention is given to projects that were
Page 4 of 29


seminal to the methodological development of the field and those that generated datasets
still widely used by contemporary scholars.
The World Event/Interaction Survey (WEIS)

The World Event/Interaction Survey Project began at the University of Southern
California under the direction of Charles McClelland as a research project on the
characteristics and processes of the international system (McClelland and Hoggard
1969). The initial WEIS dataset records the flow of action and response between
countries (as well as non-governmental actors such as NATO and the United Nations)
captured from a daily content analysis of the New York Times from January 1966 through
December 1978. This reliance on the New York Times produces a well-known bias toward
western perspectives, which was acknowledged from the outset by McClelland and his co-
authors. However, they argued that by using a single source they were better able to
remove the “noise” surrounding observations. Furthermore, the inclusion of non-state
actors raises important methodological issues with regard to the basic unit of analysis.
This question has taken on increased salience with the rise in concern about terrorist
activities by non-state international entities.
The basic unit of analysis in the dataset is the interaction, which is simply a verbal or
physical exchange between nations ranging from agreements to threats to military force.
Each of these observations is coded to identify the actors, target, date, action category,
and arena. The WEIS databank also provides brief qualitative textual descriptions of each
event. These narratives provide context, which facilitates the process of identifying and
understanding outliers and applying statistical findings back to political reality – both
important for successful events analysis. The initial WEIS effort has been continuously
updated and is presently current through 1993 (Tomlinson 1993). Other projects, such as
the Kansas Event Data System, have applied WEIS coding rules to new research.
WEIS data has been widely used in the FPA literature, both by McClelland and his
students and by outsiders who took advantage of these public domain data to test their
own questions. The applications are diverse, underlining the versatility of well-designed
events datasets. Several early examples are noted by Rummel (1979): Tanter (1974) used
these data to understand the dynamics of the two major Berlin crises of the Cold War
(1948–1949 and 1961); Kegley et al. (1974) explored patterns of international conflict and
cooperation; while many others began the ongoing process of understanding the
relationships among key contextual variables such as relative development, size, and
political system, on international conflict, cooperation, and systemic stability (Rosenau
1974). Applications continue to this day. For example, Reuveny and Kang (1996B) utilized
WEIS data in their exploration of causality in the relationship between international trade
and conflict.
The Conflict and Peace Data Bank (COPDAB)

The COPDAB project was designed by Azar and colleagues (Azar 1980; 1982; Azar et al.
1972) as a longitudinal dataset of international and domestic events developed through
content analysis of daily newspapers. In an advance over WEIS methods, COPDAB data is
Page 5 of 29


drawn from a wide variety of international and regional media outlets, thereby avoiding
some potential bias issues.
COPDAB coders scored each event on a 16-point ordinal scale ranging from cooperative
interactions to full-scale violence. The resulting dataset covers the interactions of 135
countries from 1948 to 1978 and can be analyzed at levels of aggregation ranging from
the day to the year. Each record includes nine variables: date of event, actor initiating the
event, target of the event, issue area(s), contextual information about the incident, and
the source of the information about the event. The COPDAB dataset is particularly useful
for those interested in the interactions between interstate and civil conflict and
cooperation, as complementary datasets exist for both international and domestic events.
While the WEIS and COPDAB datasets are clearly conceptually related, scholars have
disagreed about their compatibility (Howell 1983; Vincent 1983; Goldstein and Freeman
1990). The underlying definitions of conflict and cooperation are quite similar; however,
coding differences introduce the potential for inconsistencies. Reuveny and Kang (1996A)
explore this issue in detail with a series of statistical tests and time-series analyses. They
argue that COPDAB and WEIS are indeed compatible for the overlapping period between
1966 and 1978. Building on this logic, they combine the WEIS and COPDAB series to
create a larger events dataset covering the period from 1948 to 1993 that is potentially
useful for scholars interested in working with a longer period of analysis.
International Crisis Behavior Project (ICB)

Although the final two projects outlined here (the International Crisis Behavior project
and the Correlates of War [COW] project) are often excluded from discussions of foreign
policy analysis, they are clearly a continuation of events research and are among the most
frequently updated and widely used events datasets. The distinctive feature of the ICB
and COW datasets is that they primarily focus on international conflict and therefore lack
the range of conflictual and cooperative events that characterize the data projects
discussed to this point. Researchers should note, however, that the ICB project does
provide some indirect data on cooperation.
Michael Brecher and Jonathan Wilkenfeld launched the International Crisis Behavior
project in 1975 with the goal of creating a comparative resource for those studying the
concept of “international crisis.” There are two defining conditions for a crisis, which are
built on work done by Azar (of the COPDAB project): “(1) a change in type and/or an
increase in intensity of disruptive, that is, hostile verbal or physical, interactions between
two or more states, with a heightened probability of military hostilities; that, in turn, (2)
destabilizes their relationship and challenges the structure of an international system –
global, dominant, or subsystem” (Brecher and Ben-Yehuda 1985).
The ICB project is congruent with many of the core concepts in FPA – for example, in the
operationalization of key elements of decision maker perception. This is perhaps
unsurprising, as many of the ICB’s primary researchers are steeped in the FPA tradition.
To take one example, Michael Brecher’s (1974) book on Israeli foreign policy decisions,
Page 6 of 29


which pre-dates his work on the ICB project, is often cited as a seminal contribution to
FPA that seeks to characterize a nation’s psychological and cultural environment as an
access point to an understanding of its foreign policy.
As of January 2009, the core systemic dataset that results from this definition codes 452
incidents from the end of World War I through 2006 (version 9.0). Each crisis is coded for
a number of variables, ranging from characterizations of decision maker perception to
operationalizations of structural and environmental factors as well as crisis
characteristics and outcomes.
The ICB project is unusual in that it proceeds simultaneously at multiple complementary

levels. There are independent actor and system level datasets that allow the researcher to
explore distinctions between systemic and national level explanations for crisis
emergence and behavior. In addition, the project provides qualitative data in the form of a
brief narrative description of each crisis, 9 in-depth volumes comprising 15 in-depth case
studies; and 14 other unpublished studies. These serve as an aid to the researcher
interested in adding additional nuance to statistical findings generated from quantitative
analysis.
Correlates of War Project (COW)

Like the ICB project, the COW project does not attempt to capture multiple tiers of
conflict and cooperation, but rather focuses on conflict. Two definitions were developed
by the COW project in the 1980s, namely, a “militarized interstate dispute” (MID), and a
“militarized interstate crisis” (MIC). The former is defined as: “[A] set of interactions
between or among states involving threats to use military force, displays of military force,
or actual uses of military force […] these acts must be explicit, overt, non-accidental, and
government sanctioned” (Gochman and Maoz 1984). This “evolves into a militarized
interstate crisis when a member of the interstate system on each side of the dispute
indicates by its actions its willingness to go to war to defend its interests or to obtain its
objectives.” These are steps two and three along a four-step ladder of growing
belligerence, beginning with an “interstate dispute” and culminating in an “interstate
war” (Leng and Singer 1988).
The majority of scholars currently working with COW events data use the MID dataset.
The current version of the dataset contains 2331 militarized disputes from 1816 to 2001
coded for duration, outcome, and level of fatality. In addition, there are several
complementary datasets on various metrics of international interaction (ranging from
alliance to power to geography) that are associated with the broader COW project and
can be easily mapped onto the MID dataset. This body of quantitative data is perhaps the
most widely used at the present time – particularly among scholars interested in conflict.
Trade Dependence and Foreign Policy Compliance
An important strand of the behavioral work of the 1970s and 1980s addressed the
relationship between trade dependence and foreign policy compliance. While this was far
Page 7 of 29


from the only research question to draw on quantitative data, the methodological
challenges that confronted it were representative of those faced by quantitative FPA in
general and are therefore worthy of some attention. Several scholars working in this area
(e.g., Richardson and Kegley 1980; Moon 1983; 1985) argued that relatively smaller and
weaker states adopt the foreign policies of their dominant trading partners. Thus,
economic dependence severely constrains the independent decision making of leaders in
states that are economically reliant on larger patrons. However, consensus on this
conclusion was elusive, in large part because of how difficult it is to measure the two key
concepts – dependence and compliance. The inevitable result was that discussion of the
relationship became bogged down in issues of definition and operationalization. This is
symptomatic of a larger issue in the quantitative study of foreign policy. Because the
operationalization of the amorphous concepts in foreign policy necessitates discretion
from the researcher, it is easy to critique the underlying assumptions that gave rise to the
data, not to mention the model. Furthermore, if more than one scholar takes on a
question in FPA, they typically settle on different operationalizations of the same
underlying phenomenon. A high profile example of this trend can be found in the
proliferation of events datasets on conflict and cooperation that has already been
discussed. The unfortunate result is that many studies are not comparable or cumulative
to the degree we find in the hard sciences.
Events Data – Methodological Challenges
Events data analysis poses a number of methodological challenges that should be taken
into account by those analyzing foreign policy. The first of these issues relates to the very
core of the events data endeavor – that is, the idea that foreign policy incidents can be
reduced to a single quantifiable value. Despite the best efforts of the designers of the
data projects described here, it remains difficult to effectively accomplish a cardinal or
even ordinal ranking of disparate foreign policy events. However, many of the statistical
approaches widely used in political science require cardinal level data, or at least data
spaced at even thresholds. As a result, those seeking to generate statistical models of
events data need to be particularly careful to apply methods that rely upon defensible
assumptions about the nature of the underlying data.
Researchers should also be aware of methodological issues that may arise from the
relative sparseness of positive observations in events data. The degree to which this is a
problem depends on the type of model and the level of aggregation that is used, but if one
considers the daily probability of a foreign policy event it is apparent that null
observations would dominate the dataset. King and Zeng (1999; 2001) demonstrate that
bias and inappropriately inflated statistical significance may arise in models of zero-
inflated data. This is particularly problematic in instances where these null data contain
no real information. There are several potential solutions to this problem should it arise.
Tomz, King, and Zeng (1999) suggest a rare events correction for logistic analysis, which
they have made available as an addition to the widely used STATA software. A less
sophisticated check for rare events bias is to simply drop a random subset of null
Page 8 of 29


observations in order to confirm that findings derived from the remaining sample are
consistent with the original result.
The non-independence of foreign policy events presents an additional methodological

challenge. Non-independence simply means that positive foreign policy interactions tend
to contribute to future positive interactions, while negative events are associated with
subsequent negative events. At first appearance this might seem obvious, but this reality
undercuts an assumption of independence that underpins most statistical models used in
quantitative foreign policy analysis. Beck, Katz, and Tucker (1998) did much of the work
that brought this problem to the attention of the discipline and they suggest a solution
that entails generating a natural cubic spline with knots at the first and second
derivative.
FPA scholars working with events data should also guard against selection bias
(sometimes referred to as selection effects) when designing research, as inattention to
this methodological challenge can significantly skew findings from both quantitative and
qualitative tests. Selection bias typically arises from pre- or post-sampling that
preferentially includes or excludes a particular type of observation from the sample that
is subsequently used in testing. This is particularly easy to do when working with data on
foreign policy because it is relatively easy to identify events, but difficult to tease out non-
events. The trouble is that without an accurate characterization of non-events it is
impossible to say anything about the causes or incidence of the events. To take one
prominent recent example of this methodological challenge, Robert Pape’s recent work
on the causes of suicide terror (2005) has come under fire for “sampling on the
dependent variable” (Ashworth et al. 2008). Because Pape limits his sample to incidents
of suicide terror, he effectively leaves out the instances in which such attacks did not
occur. As a result, his research design prevents him from effectively speaking of when
suicide terror does and does not occur.
Beyond issues related to the application of statistical methods to events data, there is an
additional conceptual concern regarding the unit of analysis that should command
attention from foreign policy researchers. Because FPA concerns the foreign policy of
states, but sees this policy as emerging from the actions of individuals, traditional units of
analysis are blurred. The foreign policy event is the result of the interaction and interplay
between leaders, organizations, institutions, and states; however, many of the
microfoundational theories that underpin the FPA endeavor are cast at the level of the
individual decision maker. As a result, events analysis brings with it the nascent challenge
of explaining how individual actions aggregate to the foreign policy actions of states. To
put the issue more succinctly, while FPA theories distinguish themselves from mainline
international relations by opening the black box of the state, the empirical data collected
by scholars interested in events analysis typically returned to the state as the central unit
of analysis.
There are also very practical concerns to bear in mind – simple tasks related to data
manipulation remain some of the primary challenges confronting researchers interested
Page 9 of 29


in working with events data. It can be a nontrivial task to gather and combine data on
foreign policy events with the various explanatory and control variables that are required
for regression analysis. Researchers confronted with these difficulties should be aware of
the EUGene software developed by Scott Bennett and Allan Stam (2000). EUGene is a
basic data management tool that simplifies quantitative analysis of foreign policy
interactions. The software offers several advantages. First, it allows for relatively easy
transition between commonly used units of analysis – country–year, dyad–year, and
directed dyad–year. Second, the software is capable of easily combining many of the
events datasets discussed here with basic demographic and geopolitical data including
data uploaded by the user.
Finally, there is the issue of collecting new events data. The substantial early investments
in projects like WEIS and COPDAB were made at the high point of governmental and
institutional enthusiasm for events datasets – both datasets were products of the National
Science Foundation’s well funded Data Development for International Research (DDIR)
project. However, DDIR funding and government and private support for events data
collection projects in general declined markedly by the mid-1990s. While this decline had
many causes, it was in part brought on by the difficulties that comparative foreign policy
had delivering on its early promise. It proved far more challenging than expected to build
policy relevant quantitative models with predictive capacity. The relative decline in
interest on the part of traditional funding sources raises the issue of how new events data
might be generated. Computer coding of electronically stored sources, which will be
discussed in greater detail later in this essay, has emerged as one way to address this
dilemma.
Qualitative Methods of Foreign Policy Analysis

The behavioral revolution and Cold War politics proved fertile ground for the emergence
of FPA. However, the first major challenge for the young field also stemmed from this
dual heritage. The problem was that these divergent intellectual pedigrees gave rise to
methodological requirements that were at times mutually exclusive – on the one hand, an
imperative from behavioralism for generalizability, and, on the other, a low tolerance for
error on the part of Cold Warriors seeking to immediately inform policy with scientific
findings. The emerging recognition of this tension and the seemingly unavoidable high
error terms in quantitative models of foreign policy brought an end to the exuberance
among academics and the US government for quantitative, events-driven foreign policy
research. Policy makers backed away from direct involvement in the FPA endeavor, while
academics tempered their commitment to events data and quantitative methods. What
emerged was a second generation of FPA methodology, one that largely abandoned
universalized theory-building in favor of historical methods and qualitative analysis
(Neack et al. 1995).
The primary weapon in the arsenal of second-generation FPA researchers is the case
study. However, this transition should not be viewed as a complete departure from that
Page 10 of 29


which came before it. Many of these scholars place particular emphasis on developing
case study methodologies driven by social science principles, with the explicit goal of
building techniques that provide intellectual rigor comparable to that of quantitative
approaches. The result has been a robust discussion of the role and execution of
qualitative methods.
It is admittedly artificial to divide methods of foreign policy analysis by “generation,” as

this implies clean transitions that are in reality far more blurred. While the concept of
generational change is useful for understanding the broad developments in the field, the
reader should be aware that there are many exceptions to the general rule. Alongside
second generation case studies were a wide range of quantitative approaches that, while
often abandoning the drive toward universalized theory that characterized previous work
in comparative foreign policy, stressed both the outputs and the outcomes of foreign
policy processes and actions. Similarly, careful qualitative analysis of foreign-policy
decision making has always been an element of foreign policy analysis, and therefore
cannot only be considered to have followed sequentially on the quantitative work done in
comparative foreign policy (although it did take on renewed prominence).
Case Study Analysis
There is no shortage of examples of the excellent use of case study methodology in

foreign policy analysis. Graham Allison’s Essence of Decision (1971) is often cited as a
seminal piece of research in this area with an innovative methodological approach. While
Allison’s volume is concerned with a single incident – the Cuban missile crisis – the book
is not a single case study, but rather three. Allison explored the US decision making
process in the context of three competing explanatory theories: a rational actor model, an
organizational process model, and a government politics model. Each of these three
explanatory models receives independent analysis in a separate section of the book.
Allison (1971:258) argues that these three models combine to provide a clear
understanding of decision making in the context of the Cuban Missile Crisis: “Model I
fixes the broader context, the larger national patterns, and the shared images. Within this
context, Model II illuminates the organizational routines that produce the information,
options, and action. Model III focuses in greater detail on the individuals who constitute a
government and the politics and procedures by which their competing perceptions and
preferences are combined.”
Another important strand of qualitative foreign policy analysis draws on work from
political psychology to theoretically inform case study analysis of the foreign policy
decision making process. These efforts began with “operational code analysis,” which
involves determining how decision makers’ core beliefs shape their foreign policy
reactions (George 1969; Holsti 1970). Operational codes include decision makers’ beliefs
about the likelihood of violence, their ability to shape or prevent it, as well as leadership
strategies and styles.
Page 11 of 29


Robert Axelrod applies a related technique, termed cognitive mapping, to understand the
influence of leadership beliefs on foreign policy interactions. Cognitive mapping entails
defining a decision maker’s stated goals and then determining the causal linkages
between these goals as a way of predicting likely behavior. Several applications of this
technique can be found in an edited volume titled Structure of Decision (Axelrod 1976). A
more recent application of cognitive mapping can be found in Johnston’s (1995) work on
Chinese–American relations.
This early work developed into a substantial body of foreign policy analysis based more
broadly on the psychology of decision makers, a method that figures prominently in
analyses conducted at the individual level. Larson (1985) is a leading example of this sort
of scholarship. In her book, Origins of Containment, she traces the path of Cold War
politics in the context of the cognitive psychology of American policy makers.
A great deal of work has been done in recent years to improve and formalize case study
methodology. One such volume, King, Keohane, and Verba’s Designing Social Inquiry
(1994), has been influential (and controversial) enough that it is often referred to simply
by the initials of its authors – KKV. King, Keohane, and Verba draw on their diverse
methodological backgrounds to argue that the core logic of causal inference and control
should apply as much to qualitative work as it does to quantitative research. They suggest
that, by applying the logic of statistics, it is possible to produce theoretically robust and
generalizable results while increasing certainty in the validity of qualitative findings.
Bennett and George’s (2005) more recent work on case study analysis has also emerged
as an important contribution to the development of robust qualitative methods. This book
lays out methods for designing case studies that are maximally useful for the formulation
of policy, which remains a fundamental goal of foreign policy analysis. Bennett and
George suggest greater emphasis on within-case analysis, process tracing, and theory
building. While these suggestions differ markedly from those of KKV, the underlying goal
is quite similar – to create scientific case studies from which lessons can be
systematically drawn. In this sense, both volumes speak convincingly to the
aforementioned tension between nuance and generalizability that plagues methods of
foreign policy analysis.
This issue of generalizability has developed into the core methodological challenge
surrounding case study analysis both in foreign policy analysis and in political science
more generally. While systematic knowledge of foreign policy interactions does not
necessarily require the numerical comparability that comes with quantitative research,
some degree of generalizability remains important to the independent identity of foreign
policy analysis, as it is this forward-looking element that separates the sub-discipline from
diplomatic history. However, comparisons across cases are difficult for two reasons. First,
case studies require such a depth of knowledge and investment of time that it is unusual
for a scholar to accomplish more than a handful of them on any given question, though
there are important exceptions (e.g., Brecher 2008). Second, the comparatively loose
structure of case studies can hinder comparison, as many analyses fail to address the
Page 12 of 29


same subjects on the same terms. One way that these challenges can be overcome is
through collaboration within a consistent framework.
An example of such collaboration can be found in a relatively recent volume edited by

Beasley et al. (2002). The volume brings together qualitative work from 15 independent
researchers systematically exploring the foreign policies of 13 states. Through the
coordinating efforts of the editor, the volume maintains a degree of comparability across
the cases while drawing on the deep knowledge of the individual contributors. As a result,
the reader is able to engage in comparative analysis within a coherent theoretical
framework, allowing for the quick identification of patterns and outliers. There are
several examples of similarly structured volumes, and they indicate an important role for
collaboration as an approach to boosting the sample sizes of qualitative analyses and
thereby the generalizability of findings. The result is “comparative foreign policy,” but of
a qualitative variety.
Another interesting solution to the issue of case comparability is found in the qualitative
research that has emerged from the qualitative side of the International Crisis Behavior
Project, which was already mentioned in the context of events data analysis. These case
studies, though they were written over many years and appear in a variety of different
outlets, follow a similar format and concern themselves with a consistent set of issues. As
a result, they are an explicitly cumulative effort. With each new case study, the body of
comparable knowledge increases and this expansion is accompanied by improvement in
the robustness of findings.
Gathering Qualitative Data
Those interested in applying case study methodology will need raw material with which
to build their analysis. For many questions, considerable ground can be covered using
basic library research techniques and secondary sources. However, some of the most
fruitful case studies (in terms of their contribution to the existing body of knowledge)
bring new information to light. There are several methods of obtaining original qualitative
data. The sections that follow will briefly discuss four methods that have become central
in foreign policy analysis: archival research, content analysis, interviews, and focus
groups.
Archival Research
Original source material can be a crucial element of a quality case study. Typically,
scholars uncover such information through archival research. Relevant foreign policy
materials are commonly found in the document collections housed in presidential
libraries, national archives, and universities. While the basic concept behind archival
research is self-explanatory, the actual process of gaining access to collections and
navigating their contents can intimidate the neophyte. There are a number of guides to
archival methods that can alleviate such anxiety. Directions for identifying and searching
appropriate archival sources as well as tips for navigating the archives themselves can be
found in Marc Trachtenberg’s (2006) recent volume on methods – Appendix II will be of
Page 13 of 29


particular interest to those seeking to utilize archival methods. Hill (1993), Larson (2001),
and Lustick (1996) provide additional detail on the nuances of archival research.
Content Analysis
Content analysis is a hybrid method that has played a longstanding and important role in
quantitative and qualitative foreign policy analysis. The section of this essay on events
data already discussed the ways in which content has been used to generate quantitative
data for statistical analysis. For example, some of the earliest approaches to events data
generation coded the content of elite communication (Winham 1969). However, more
detailed content analyses have also been used to generate the raw material for case
studies or other qualitative analyses. Ole Holsti (1969) was a pioneer of this method,
while, more recently, Steve Walker and his students at Arizona State have developed a
typology and quantitative content analysis scheme for operational code analysis (Walker
et al. 1998). Those interested in additional detail on the mechanics of content analysis
should consult Weber (1985), Neuendorf (2002), and West (2001).
Interviews
Because the role of the individual figures so prominently in foreign policy analysis,
interviews can be a particularly valuable method for accessing information about the
mechanics of the decision making process. Interviews enable FPA scholars to delve
deeply into the idiosyncrasies of the foreign policy process, gleaning deep insights from
decision makers and those around them. Over time, FPA scholars have developed a robust
set of interview methods designed to enable researchers to maximize the acquisition of
information without introducing biases into findings.
There are a number of excellent examples of innovative interview methods in foreign

policy analysis, which can serve as models for those interested in interview research.
Prime among them are FPA classics such as Yuen Foong Khong’s Analogies at War (1992).
Based on a series of interviews with senior officials (and archival research), Khong argues
that leaders routinely reference the past when making foreign policy decisions and that
this cognitive bias can profoundly alter decision making. Schoutlz (1987) does similar
interview work in the context of US policy toward Latin America. More recently, Silber
and Little (1995) draw on a series of interviews to uncover the foreign policy interactions
at play in the collapse of the former Yugoslavia. Berg (2001), Brenner et al. (1985),
McCracken (1988), Mishler (1986), and Seidman (1998) provide useful, in-depth tutorials
on interview methods.
Focus Groups
Focus group research is a derivative of interview methodology in which the researcher
attempts to facilitate an organized discussion among the participants. In foreign policy
analysis this typically takes the form of a meeting of experts in a particular foreign policy
area, or participants in a prior foreign policy decision. Focus group methods can be
particularly informative because the emerging consensus that comes from such
discussions pools the knowledge of the participating individuals and therefore can
overcome some of the potential biases of recollection and self-inflation that accompany
Page 14 of 29


individual interviews. However, concerns arise as well, due to some of the very same
pathologies that FPA scholars have identified in the context of group decision making. For
example, Janis’s (1972) concept of groupthink can take hold in such settings, with focus
group members avoiding controversy and settling instead on a comfortable consensus,
even if this consensus is out of step with reality. Along similar lines, the value of elite
focus groups can suffer due to deference to higher-ranking participants and domination
of the discussion by more talkative individuals who might overshadow important
contributions by those less inclined to assert themselves (Krueger 2000).
Third Generation Methods of Foreign Policy

Analyses
Neack, Hey, and Haney’s (1995) concept of generational change, to which this review has
adhered thus far, captures only part of the methodological richness of FPA. There have
long been methods of foreign policy analysis that fall outside this strict quantitative/
qualitative divide, and there has been considerable recent growth in these alternative
methods. Meanwhile, the distinction between quantitative and qualitative approaches to
FPA has become increasingly blurred as the relative advantages of each approach have
become more widely recognized. These events auger the arrival of a third generation of
FPA scholarship that combines innovative quantitative and qualitative methods, thereby
bridging the internal contradictions that split the second wave from the first and unifying
a variety of methods of foreign policy analysis.
Several methods of foreign policy analysis are available to aspiring third generation
foreign policy analysts seeking to move beyond events data and case studies including:
computer assisted coding, experiments, simulation, surveys, network analysis, and
prediction markets. The sections that follow will briefly introduce each of these methods,
though the list is by no means exhaustive.
Machine Coding
Computer assisted coding of electronically stored information offers several advantages

and represents an important methodological innovation that is likely to play an
increasingly significant role in the future of foreign policy analysis. First, machine coding
can be more reliable than human coding simply because it removes the possibility of
individual error and the resulting questions of intercoder reliability. Second, machine
coding is extremely rapid. Where earlier events datasets were generated over periods of
many years, computers can sift through huge quantities of data in minutes. The result is
that machine coding greatly reduces the cost of events data generation – effectively
bringing control over such data to the masses (Gerner et al. 1994; Schrodt and Gerner,
1994). However, the obvious benefits of machine coding are accompanied by two
important caveats: the initial programming that creates the coding rules must be
accurate, and the raw data must exist in a machine readable format (Gerner et al. 1994).
Page 15 of 29


Advocates of human coding often counter that the low cost and speed of machine coding
are accomplished at the expense of accuracy and nuance.
At present, the best example of a machine coded events project is the Kansas Event Data
System (KEDS). This project is among the most active events datasets, due in part to the
relatively low cost and speed of generating data in this manner. KEDS provides a
computer program that enables users to specify and create personalized events datasets
with a variety of output options. The researchers on the KEDS team use this software to
code news reports and generate political event data focusing on the Middle East, the
Balkans, and West Africa; however, this approach can be extended to other regions or the
international system as a whole.
The machine coding community, including members of the KEDS project, is particularly
interested in predictive models built on the unique capacities of this technology (Schrodt,
1979; 1994; Gerner et al. 1994; Schrodt and Gerner 2000; Shellman FORTHCOMING).
Machine coding not only partially circumvents the need for large financial investments in
events data by reducing the required labor and time, but also has the potential to address
some of the concerns about the lack of predictive capacity that caused the decline in
external funding in the first place. Because machine coding concentrates the researcher’s
effort on developing decision rules rather than on the coding itself, once underway these
programs can generate empirical data in real time. Such models that draw on
continuously updated data effectively serve as early warning systems capable of
identifying when political phenomena of interest are likely to occur. For example,
Shellman and Stewart use machine coding to predict incidents of forced migration, which
they applied with some success in Haiti (Shellman and Stewart 2007). This particular
application of events data remains at the cutting edge of the FPA literature and will likely
continue to be a productive avenue for future research.
Experiments and Simulation in Foreign Policy Analysis
Like all social science, foreign policy analysis struggles methodologically with the issues
of control and causality. The quantitative and qualitative methods already discussed took
hold in foreign policy analysis in part because the gold standard of the scientific method –
experimental control – is typically off limits either for practical or for ethical reasons.
However, with careful attention to design and feasibility, there are applications for
experimental methods in the study of foreign policy, and where there are not, researchers
have begun to turn their attention to simulation, which can achieve some of the same
objectives. To take one recent example, Christensen and Redd (2004) examine how the
context of foreign-policy decision making affects choice and assess this relationship in a
controlled experiment conducted on undergraduates. They find that, at least in this
context, the way in which information is presented directly affects the decision maker’s
evaluation.
In recent years the nuts and bolts of experimental methods have drawn increasing
attention. Along these lines, McDermott (2002) provides an interesting discussion of the
Page 16 of 29


origins and practice of experimental methods in political science, as well as the unique
challenges this approach presents. One such challenge that should be considered
carefully by those designing experiments meant to speak to foreign policy behavior is the
trade-off between internal and external validity in experiments. Internal validity indicates
that the proposed relationship between the independent and dependent variables is the
true causal one. When such validity is high it means that extraneous variables and
alternative explanations have been ruled out. While typically very difficult to achieve in
the social sciences, high internal validity results from proper randomization in an
experiment. External validity speaks to the degree to which a proposed relationship is
generalizable to a broader set of cases or the world at large. Thus, experimental methods
are powerful because they are high in internal validity; however, a leap occurs when we
attempt to generalize experimentally derived results to actual political behavior.
This leap can be particularly worrisome when it is from an experimental finding

generated from a non-elite individual – for example, an undergraduate student as was the
case in the Christensen and Redd study – to a foreign policy decision maker. In such
cases, the assumption of external validity may not be reasonable. Mintz, Redd, and Vedlitz
(2006) explore this issue in detail, replicating an experiment on the subject of
counterterrorism with a group of college student and a group of military officers. The
authors find significant differences between these groups, suggesting that experimental
subjects cannot be expected to play the role of foreign-policy decision makers without
careful regard for their actual background. However, while these scholars argue that
average individuals can tell us very little about the behavior of elites, they do find it more
acceptable to use subjects like students as a sample of the public at large.
Simulation, a close relative of experimental methods, has its roots in the longstanding
practices of war gaming and diplomatic analysis. However, recent efforts in this area
draw extensively on advances in computing power and the internet. Research in this area
builds on early work by the Inter-Nation Simulation (INS) project (Guetzkow et al. 1963),
and slightly later efforts by Hermann (1969) and Alker and Brunner (1969).
The International Communication and Negotiation Simulations (ICONS) project is an

ongoing extension of this early work that allows political practitioners and students to
develop decision making and foreign policy skills through computer aided interactive
simulation. Jonathan Wilkenfeld and Richard Brecht developed ICONS in the 1980s,
building on Noël’s (1969) early POLIS simulations. As presently formulated, the ICONS
project is more about training than research, but the technique presents an intriguing
methodological opportunity for those interested in testing theories of foreign policy
interactions in a controlled environment.
Survey Research in Foreign Policy Analysis
When it is focused at the elite level, as it often is, survey research in foreign policy
analysis directly relates to the previously discussed interview methods. This stands in
some contrast to the way in which survey research is conducted in other areas of political
Page 17 of 29


science. For example, in American politics there is a long tradition of survey research
designed to pinpoint public opinion on a myriad of topics. In order to accomplish this,
researchers are obliged to reach as representative a sample of the population as possible.
In contrast, FPA’s focus on elite perception and behavior as a determinant of foreign
policy leads to the wider usage of elite interviews.
While surveys lack the depth of an interview, they offer the corresponding advantage of
breadth. First, by aggregating information from a more significant number of sources, a
survey can minimize some of the idiosyncratic error that can plague interview
methodology. Second, in a survey analysis it is easier to control for secondary variables
that might influence the recollection or reporting of subjects. Finally, surveys can both
contribute to qualitative analysis, and serve to generate high-quality data for aggregate
analysis.
Holsti and Rosenau’s (1979; 1980) work on post-Vietnam attitudes is an excellent example
of what can be accomplished in foreign policy analysis with elite surveys. Holsti and
Rosenau were interested in the degree to which historical experience altered the
perceptions and beliefs of opinion leaders and decision makers. Their expectation was
that the Vietnam conflict significantly altered the perspective of those who drew their
primary experience from that conflict rather than World War II. To answer this question
they extensively surveyed groups that they believed to comprise the national leadership
structure – military personnel, foreign service officers, business executives, labor leaders,
clergy, media, etc. – and found significant differences between occupations and within
generations.
Surveys can be particularly valuable when conducted repeatedly over several years, as
this allows for longitudinal analysis – something that is crucial if one is interested in
changes over time. Both the Chicago Council on Foreign Relations and the Pew Research
Center for the People and the Press conduct quadrennial surveys of government,
academic, military, religious, and scientific “influentials” in order to measure the content
of and changes in elite opinion. These surveys, and others that could be conducted along
similar lines, are an underutilized resource for foreign policy analysis. Presser et al.
(2004) and Rea and Parker (2005) are useful resources for those seeking additional detail
on the mechanics of survey research and questionnaire design.
Network Analysis
FPA scholars can also benefit from the recent explosion of interest among political
scientists in network analysis. Social network analysis, which is simply the mapping and
measuring of relationships among entities in a complex system, is a useful tool for
modeling foreign policy relationships because it incorporates both bilateral connections
and wider connections among the larger group. Because of this, the technique analysis
allows FPA scholars to understand relational data – the contacts, ties, connections, and
transfers between decision makers that cannot be cleanly reduced to properties of the
leaders themselves (Scott 1991). Furthermore, a network theoretic framework
Page 18 of 29


consistently captures the role of third parties in foreign policy interactions, which prove
to be crucial to understanding outcomes.
Relational approaches have long been an underlying element in the study of foreign
policy. For example, Anne-Marie Slaughter (2004) writes on the relationship between elite
networks and international conflict. However, quantitative social network analysis first
began to make significant inroads into political science in the 1990s primarily through
the study of “policy networks” (Marin and Mayntz 1992; Marsh and Rhodes 1992), though
there are earlier, pioneering examples (e.g., Eulau and Siegel 1981; Tichy et al. 1979).
These studies, as well as later work in international relations (e.g., (Hammarström and
Birger 2002; Wilkinson 2002; Montgomery 2005; Heffner-Burton and Montgomery 2006;
Maoz 2006; Ward 2006), provide models for future work with foreign policy networks. In
short, relational thinking and social network analysis have already contributed to the
clarification of a number of puzzles in political science and present a potentially powerful
way of approaching foreign policy analysis.
Prediction Markets
Prediction markets are information exchanges built to generate forecasts using a price
mechanism. Futures generated from predictions of upcoming events are traded, such that
their value is tied to a particular outcome. The result of this arrangement is that the
market prices of these futures can be interpreted as the predicted probability of that
outcome. There is a significant body of research that establishes the ability of markets to
reduce error in predictions. By aggregating the bets of many individuals, these markets
effectively use the price setting mechanism to uncover the consensus about a future
foreign policy event in much the same way that the stock market predicts the economic
performance of a company or oil futures respond to the expected scarcity of that
resource. Pennock et al. (2001) demonstrate that in many cases prediction markets
systematically outperform the estimates of even the best individual analysts. There are
only a few examples of longstanding prediction markets that handle political futures.
These include Intrade, which floats, among many other things, a diverse group of political
contracts, and the longer running Iowa Electronic Market, which is an academically
oriented project designed for evaluating the probability of election outcomes.
Prediction markets have been applied sparingly in international relations and foreign
policy analysis, but have tremendous potential for future application because they offer
an interactive mechanism with which individual foreign policy experts can aggregate
their knowledge and opinions. Interestingly, given the methodological diversity that
characterizes FPA as a sub-discipline, the method by which each expert who trades
futures on a prediction market reaches his or her own conclusion is irrelevant. Thus, a
prediction market can provide an alternative way to combine and generalize both deep
qualitative knowledge and quantitative findings. Furthermore, this approach presents a
novel way of dealing with error and uncertainty.
Page 19 of 29


Prospective researchers in this area should note that some early applications of this
approach have not gone smoothly. The Defense Advanced Research Projects Agency
(DARPA) recently abandoned a promising plan to use a futures market to forecast the
probability of important foreign policy events such as regime change and terrorist attacks
when the media picked up on the program and it became controversial. Despite a robust
literature on the efficacy of such markets, politicians and segments of the public seized
upon the effort as being unethical or even nonsensical (Looney 2003). The unwanted
attention led DARPA, which usually operates well beneath the public radar, to cancel the
project almost immediately. It remains an open question whether this approach will
become more politically feasible – seemingly a necessity because these markets generally
require a significant initial investment, presumably by a government or university.
However, private markets such as Intrade, which is a for-profit enterprise, seem to be a
plausible alternative. Foreign policy futures, such as the probability of an Israeli attack on
Iran, are traded regularly on Intrade and provide useful information about expectations.
Moreover, futures on the outcome of the last presidential election vied with polling data
for public and media attention in the lead up to the 2008 US presidential election
suggesting that familiarity with these markets may be rising.
Remaining Methodological Challenges

Methods of foreign policy analysis have developed markedly over the past few decades,
but challenges remain. An unavoidable tension persists between the accuracy needed for
policy relevance and the scope needed for generalizability. As the grand theories of
foreign policy interaction motivated those who launched the FPA enterprise proved
elusive, the discipline increasingly turned to the nuanced examinations of cases.
However, if taken too far this trend is a threat to the unique identity of FPA because it
blurs the distinction with longstanding traditions of historical analysis. This survey of
available methods suggests that a partial solution to this dilemma lies in bringing
quantitative analysis and underutilized “third generation” methods back into the FPA fold
by reintegrating them into the well-developed qualitative tradition. The goal should be to
develop a healthy mix of methods that applies each approach to the questions which each
is best equipped to address.
Additional attention should also be given to determining the degree to which current
methods of foreign policy analysis allow predictive or prescriptive conclusions. In recent
years, enthusiasm for FPA has been fueled in part by the failure of most international
relations scholarship to accurately foresee key events in the international system –
specifically the decline of the Soviet Union and the end of the Cold War. The argument is
made that Cold War politics, because they were in some sense stable or at equilibrium,
were better suited to elegant and parsimonious models of the systemic behavior of state
actors. In contrast, the more chaotic world we presently inhabit is characterized by
fluidity driven by human agents and therefore is best understood using the methods of
foreign policy analysis (Hudson and Vore 1995). This is a reasonable hypothesis; however,
prediction is a difficult game in the social sciences and it remains unclear whether FPA is
Page 20 of 29


indeed superior in this arena. In short, with a few notable exceptions such as the KEDS
project, methods of foreign policy analysis lack predictive capacity and, when they are
able to predict, are often unable to clearly state the degree of certainty surrounding these
forecasts. More can and should be done to improve this capacity.
Foreign policy analysts should also give deeper consideration to the issues that
accompany the choice of the unit of analysis in their models. FPA derives much of its
explanatory power from its ability to speak to the individual’s role in the foreign policy
process, but the dependent variables that these efforts attempt to explain are often the
interactions between states. The result is a gap in our understanding of the process of
aggregation by which the behavior of leaders results in the actions and reactions of
states. This aggregation problem is widely noted, but additional work is required to
complete our understanding of this element of the foreign policy process. Improvements
in this linkage between theory and test, as well as a consistent unit of analysis (individual
or foreign policy event) are particularly crucial for robust quantitative analysis, as it is in
part the inability of the subfield to resolve this basic issue that stifled earlier research on
events data.
Finally, more must be done to reengage foreign policy analysis with the core of
international relations research. FPA scholars typically claim the first and second image
as their domain, but fail to engage with those in mainline international relations who also
work in this area. In the lead essay of the first issue of Foreign Policy Analysis, Valerie
Hudson (2005) convincingly makes the case that FPA has the potential to reshape the
entire discipline of international relations by focusing attention on the workings of the
fundamental unit of analysis – the political decision maker. However, despite the call to
arms, more often than not FPA scholars labor in relative isolation. Some of these divisions
emerge from methodological issues and can therefore be resolved.
In sum, the future of foreign policy analysis appears to be bright. There is reason to
believe that longstanding methodological battles that characterized it are drawing to a
close with the recognition that multiple methods have their place in the study of foreign
policy. In addition, new methods and questions are emerging that are likely to contribute
to our understanding of the foreign policy process.
References
Alker, H.R., and Brunner, R.D. (1969) Simulating International Conflict – Comparison of 3
Approaches. International Studies Quarterly 13 (1), 70–110.
Allison, G.T. (1971) The Essence of Decision: Explaining the Cuban Missile Crisis. Boston:
Little, Brown.
Ashworth, S., Clinton, J.D., Meirowitz, A., and Ramsay, K. (2008) Design, Inference, and
the Strategic Logic of Suicide Terrorism. American Political Science Review 102 (2), 269–
73.
Page 21 of 29


Axelrod, R. (1976) Structure of Decision: The Cognitive Maps of Political Elites.

Princeton: Princeton University Press.
Azar, E.E. (1980) The Conflict and Peace Data Bank (COPDAB) Project. Journal of Conflict
Resolution 24, 143–52.
Azar, E.E. (1982) The Codebook of the Conflict and Peace Data Bank (COPDAB). College
Park: Center for International Development, University of Maryland.
Azar, E.E., Cohen, S.H., Jukam, T.O., and McCormic, J.M. (1972) Problem of Source
Coverage in Use of International Events Data. International Studies Quarterly 16 (3),
373–88.
Beasley, R.K., Kaarbo, J., Lantis, J.S., and Snarr, M.T. (2002) Foreign Policy in Comparative
Perspective: Domestic and International Influences on State Behavior. Washington, DC:
CQ Press.
Beck, N., Katz, J.N., and Tucker, R. (1998) Taking Time Seriously: Time-Series-Cross-
Section Analysis with a Binary Dependent Variable. American Journal of Political Science
42 (4), 1260–88.
Bennett, A., and George, A.L. (2005) Case Studies and Theory Development in the Social
Sciences. Cambridge: MIT Press.
Bennett, D.S., and Stam, A. (2000) EUGene: A Conceptual Manual. International

Interactions 26, 179–204.
Berg, B.L. (2001) Qualitative Research Methods for the Social Sciences, 4th edn. Boston:
Allyn and Bacon.
Brecher, M. (1974) Decisions in Israel’s Foreign Policy. London: Oxford University Press.
Brecher, M. (2008) International Political Earthquakes: Crises and Conflicts Before,

During and After the Cold War. Ann Arbor: University of Michigan Press.
Brecher, M., and Ben-Yehuda, H. (1985) System and Crisis in International Politics.
Review of International Studies 11 (1), 17–36.
Brecher, M., Steinberg, B., and Stein, J. (1969) A Framework for Research on Foreign
Policy Behavior. Journal of Conflict Resolution 13 (1), 75–94.
Brenner, M., Brown, J., and Canter, D. (eds.) (1985) The Research Interview. London:
Academic Press.
Burgess, P.M., and Lawton, R.W. (1972) Indicators of International Behavior: An

Assessment of Events Data Research. Beverly Hills: Sage.
Page 22 of 29


Christensen, E.J., and Redd, S.B. (2004) Bureaucrats Versus the Ballot Box in Foreign
Policy Decision Making: An Experimental Analysis of the Bureaucratic Politics Model and
the Poliheuristic Theory. Journal of Conflict Resolution 48 (1), 69–90.
Eulau, H., and Siegel, J.W. (1981) Social Network Analysis and Political Behavior: A
Feasibility Study. The Western Political Quarterly 34 (4), 499–509.
George, A.L. (1969) Operational Code – Neglected Approach to Study of Political Leaders
and Decision-Making. International Studies Quarterly 13 (2), 190–222.
Gerner, D.J., Schrodt, P.A., Francisco, R.A., and Weddle, J.L. (1994) Machine Coding of
Event Data Using Regional and International Sources. International Studies Quarterly 38
(1), 91–119.
Gochman, C.S., and Maoz, Z. (1984) Military Interstate Disputes, 1816–1976: Procedures,
Patterns, and Insights. Journal of Conflict Resolution 28 (4), 585–615.
Goldstein, J.S., and Freeman, J. (1990) Three Way Street, Strategic Reciprocity in World
Politics. Chicago: University of Chicago Press.
Guetzkow, H., Alger, C.F., Brody, R.A., Noel, R.C., and Snyder, R. (1963) Simulations in
International Relations: Developments for Research and Teaching. Englewood Cliffs:
Prentice-Hall.
Hammarström, M., and Birger, H. (2002) The Diffusion of Military Intervention: Testing a
Network Position Approach. International Interactions 28, 335–77.
Heffner-Burton, E., and Montgomery, A.H. (2006) Power Positions: International

Organizations, Social Networks, and Conflict. Journal of Conflict Resolution 50 (1), 3–27.
Hermann, C. (1969) Crises in Foreign Policy: A Simulation Analysis. Indianapolis: Bobbs

Merrill.
Hill, M.R. (1993) Archival Strategies and Techniques. Beverly Hills: Sage Publications.
Holsti, O.R. (1969) Content Analysis for the Social Sciences and Humanities. Reading:
Addison Wesley.
Holsti, O.R. (1970) The “Operational Code” Approach to the Study of Political Leaders:
John Foster Dulles’ Philosophical and Instrumental Beliefs. Canadian Journal of Political
Science / Revue Canadienne de Science Politique 3 (1), 123–57.
Holsti, O.R., and Rosenau, J.N. (1979) Vietnam, Consensus, and the Belief Systems of
American Leaders. World Politics 32 (1), 1–56.
Holsti, O.R., and Rosenau, J.N. (1980) Does Where You Stand Depend on When You Were
Born? The Impact of Generation on Post-Vietnam Foreign Policy Beliefs. Public Opinion
Quarterly 44 (1), 1.
Page 23 of 29


Howell, L.D. (1983) A Comparative-Study of the WEIS and COPDAB Data Sets.
International Studies Quarterly 27 (2), 149–59.
Hudson, V.M. (2005) Foreign Policy Analysis: Actor-Specific Theory and the Ground of
International Relations. Foreign Policy Analysis 1 (1), 1–30.
Hudson, V.M., and Vore, C.S. (1995) Foreign Policy Analysis Yesterday, Today, and
Tomorrow. International Studies Quarterly 39, 209–38.
Janis, I.L. (1972) Victims of Groupthink. Boston: Houghton Mifflin Company.
Johnston, A.I. (1995) Cultural Realism: Strategic Culture and Grand Strategy in Chinese
History. Princeton: Princeton University Press.
Kegley, C., Jr, Salmore, S.A., and Rosen, D.J. (1974) Convergences in the Measurement of
Interstate Behavior. In P.J. McGowan (ed.) Sage International Yearbook of Foreign Policy
Studies, vol. 2. Beverly Hills: Sage Publications, pp. 309–39.
Khong, Y.F. (1992) Analogies at War: Korea, Munich, Dien Bien Phu, and the Vietnam
Decisions of 1965. Princeton: Princeton University Press.
King, G., Keohane, R.O., and Verba, S. (1994) Designing Social Inquiry. Princeton:
Princeton University Press.
King, G., and Zeng, L.C. (1999) Logistic Regression in Rare Events Data. At http:/
gking.harvard.edu/files/0s.pdf, accessed July 2009.
King, G., and Zeng, L.C. (2001) Explaining Rare Events in International Relations.
International Organization 55 (3), 693–715.
Krueger, R. (2000) Focus Groups: A Practical Guide for Applied Research, 3rd edn.
Thousand Oaks: Sage Publications.
Larson, D.L. (2001) Sources and Methods in Cold War History: The Need for a New
Theory-Based Archival Approach. In C. Elman and M.F. Elman (eds.) Bridges and
Boundaries: Historians, Political Scientists, and the Study of International Relations.
Cambridge: MIT Press, pp. 327–50.
Larson, D.W. (1985) Origins of Containment: A Psychological Explanation. Princeton:

Princeton University Press.
Leng, R.J., and Singer, J.D. (1988) Militarized Interstate Crises – the Bcow Typology and
its Applications. International Studies Quarterly 32 (2), 155–73.
Looney, R.E. (2003) DARPA’s Policy Analysis Market for Intelligence: Outside the Box or
Off the Wall? Strategic Insights 2 (9).
Page 24 of 29


Lustick, I.S. (1996) History, Historiography, and Political Science: Multiple Historical
Records and the Problem of Selection Bias. American Political Science Review 90 (3),
605–18.
Maoz, Z. (2006) Network Polarization, Network Interdependence, and International

Conflict, 1816–2002. Journal of Peace Research 43 (4), 391–411.
Marin, B., and Mayntz, R. (eds.) (1992) Policy Networks: Empirical Evidence and
Theoretical Considerations. Boulder: Westview Press.
Marsh, D., and Rhodes, R.A.W. (eds.) (1992) Policy Networks in British Government. New
York: Oxford University Press.
McClelland, C.A. (1970) Some Effects on Theory from the International Event Analysis
Movement. Working paper. University of Southern California.
McClelland, C.A., and Hoggard, G. (1969) Conflict Patterns in the Interactions Among
Nations. In J.N. Rosenau (ed.) International Politics and Foreign Policy. New York: Free
Press, pp. 711–24
McCracken, G. (1988) The Long Interview. London: Sage.
McDermott, R. (2002) Experimental Methodology in Political Science. Political Analysis 10

(4), 325–42.
Mintz, A., Redd, S.B., and Vedlitz, A. (2006) Can We Generalize from Student Experiments
to the Real World in Political Science, Military Affairs, and International Relations?
Journal of Conflict Resolution 50 (5), 757–76.
Mishler, E.G. (1986) Research Interviewing: Context and Narrative. Cambridge: Harvard
University Press.
Montgomery, A.H. (2005) Proliferation Determinism or Pragmatism? How to Dismantle an

Atomic Bomb Network. International Security 30 (2), 153–87.
Moon, B.E. (1983) The Foreign Policy of the Dependent State. International Studies
Quarterly 27 (3), 315–40.
Moon, B.E. (1985) Consensus or Compliance? Foreign-Policy Change and External

Dependence. International Organization 39 (2), 297–329.
Neack, L., Hey, J.A.K., and Haney, P.J. (1995) Generational Change in Foreign Policy
Analysis. In L. Neack, J.A.K. Hey, and P.J. Haney (eds.) Foreign Policy Analysis. Contiguity
and Change in its Second Generation. Englewood Cliffs: Prentice Hall.
Neuendorf, K.A. (2002) The Content Analysis Guidebook. Beverly Hills: Sage.
Noël, R.C. (1969) The Polis Laboratory. American Behavioral Scientist 12 (6), 30–35.
Page 25 of 29


Paige, G. (1968) Korean Decision, June 24–30, 1950. New York: Free Press.
Pape, R.A. (2005) Dying to Win: The Strategic Logic of Suicide Terrorism. New York:
Random House.
Pennock, D.A., Lawrence, S., Giles, C.L., and Nielsen, F.A. (2001) The Real Power of
Artificial Markets. Science 291 (5506), 987–8.
Presser, S., Rothgeb, J.M., Couper, M.P., et al. (2004) Methods for Testing and Evaluating
Survey Questionnaires. New York: Wiley.
Rea, L.M., and Parker, R.A. (2005) Designing and Conducting Survey Research: A
Comprehensive Guide. San Francisco: Jossey-Bass.
Reuveny, R., and Kang, H. (1996a) International Conflict and Cooperation: Splicing
COPDAB and WEIS Series. International Studies Quarterly 40 (2), 281–305.
Reuveny, R., and Kang, H. (1996b) International Trade, Political Conflict/Cooperation, and
Granger Causality. American Journal of Political Science 40 (3), 943–70.
Richardson, N.R., and Kegley, C.W., Jr (1980) Trade Dependence and Foreign Policy
Compliance: A Longitudinal Analysis. International Studies Quarterly 24 (2), 191–222.
Rosenau, J.N. (1966) Pre-Theories and Theories of Foreign Policy. In J. Rosenau (ed.)
Scientific Study of Foreign Policy. New York: Free Press, pp. 95–151.
Rosenau, J.N. (1968) Comparative Foreign Policy – Fad, Fantasy, or Field. International
Studies Quarterly 12 (3), 296–329.
Rosenau, J.N. (ed.) (1974) Comparing Foreign Policies: Theories, Findings, and Methods.
New York: Halsted Press.
Rummel, R.J. (1979) Understanding Conflict and War, vol. 4: War, Power, Peace. Beverly
Hills: Sage Publications.
Schoutlz, L. (1987) National Security and United States Policy Towards Latin America.
Princeton: Princeton University Press.
Schrodt, P.A. (1979) Forecasting in International-Relations – Theory, Methods, Problems,

Prospects – Choucri, N., Robinson, T.W. American Political Science Review 73(4), 1208–10.
Schrodt, P.A. (1994) Event Data in Foreign Policy Analysis. In L. Neack, P.J. Haney, and
J.A.K. Hey (eds.) Foreign Policy Analysis: Continuity and Change in its Second Generation.
New York: Prentice Hall.
Schrodt, P.A., and Gerner, D.J. (1994) Validity Assessment of a Machine-Coded Event Data
Set for the Middle-East, 1982–92. American Journal of Political Science 38 (3), 825–54.
Page 26 of 29


Schrodt, P.A., and Gerner, D.J. (2000) Cluster-Based Early Warning Indicators for Political
Change in the Contemporary Levant. American Political Science Review 94 (4), 803–17.
Scott, J. (1991) Social Network Analysis: A Handbook. London: Sage.
Seidman, I. (1998) Interviewing as Qualitative Research: A Guide for Researchers in

Education and the Social Sciences, 2nd edn. New York: Teachers College Press.
Shellman, S.M. (forthcoming) Machine Coding Nonstate Actors’ Behavior in Intrastate

Conflict. Political Analysis.
Shellman, S.M., and Stewart, B. (2007) Predicting Risk Factors Associated with Forced
Migration: An Early Warning Model of Haitian Flight. Civil Wars 9 (2), 174–99.
Silber, L., and Little, A. (1995) The Death of Yugoslavia. London: Penguin.
Slaughter, A.-M. (2004) A New World Order. Princeton: Princeton University Press.
Snyder, R.C., Bruck, H., and Sapin, B.M. (1954) Decision-Making as an Approach to the
Study of International Relations. Princeton: Princeton University Press.
Tanter, R. (1974) Modelling and Managing International Conflicts: The Berlin Crises.
Beverly Hills: Sage Publications.
Tichy, N.M., Tushman, M.L., and Fombrun, C. (1979) Social Network Analysis for
Organizations. The Academy of Management Review 4 (4), 507–19.
Tomlinson, R.G. (1993) Monitoring WEIS Event Data in Three Dimensions. In R.L. Merritt,
R.G. Muncaster, and D.A. Zinnes (eds.) International Event-Data Developments: DDIR
Phase II. Ann Arbor: University of Michigan Press, pp. 55–85.
Tomz, M., King, G., and Zeng, L.C. (1999) RELOGIT: Rare Events Logistic Regression
(Version 1.1). Cambridge. At http:/gking.harvard.edu/stats.shtml, accessed July 2009.
Trachtenberg, M. (2006) The Craft of International History. Princeton: Princeton

University Press.
Vincent, J.E. (1983) Weis Vs Copdab – Correspondence Problems. International Studies

Quarterly 27 (2), 161–8.
Walker, S.G., Schafer, M., and Young, M.D. (1998) Systematic Procedures for Operational
Code Analysis: Measuring and Modeling Jimmy Carter’s Operational Code. International
Studies Quarterly 42 (1), 175–89.
Ward, H. (2006) International Linkages and Environmental Sustainability: The

Effectiveness of the Regime Network. Journal of Peace Research 43 (2), 149–66.
Weber, R.P. (1985) Basic Content Analysis. Beverly Hills: Sage.
Page 27 of 29


West, M.D. (2001) Theory, Method, and Practice in Computer Content Analysis. Westport:
Ablex.
Wilkinson, D. (2002) Civilizations as Networks: Trade, War, Diplomacy, and Command-

Control. Complexity 8 (1), 82–6.
Winham, G.R. (1969) Quantitative Methods in Foreign Policy Analysis. Canadian Journal
of Political Science 2 (2), 187–99.
Links to Digital Materials

International Crisis Behavior Project. At www.cidcm.umd.edu/icb/, accessed July 2009.
The ICB project provides quantitative and qualitative data on international crises. The
core systemic dataset currently contains 452 incidents from the end of World War I
through 2006. The link above provides access to ICB data, codebooks, citations, and a
variety of other useful materials.
Correlates of War Project. At www.correlatesofwar.org/, accessed July 2009. The COW

project website provides widely used events data on militarized interstate disputes as
well as several other datasets that may be appropriate for FPA scholars. Data,
documentation, and codebooks are available through the link.
EUGene Software. At www.eugenesoftware.org/, accessed July 2009. The link provides

access to the EUGene software package, as well as manuals and documentation (all free
of charge). EUGene allows for relatively easy transition between commonly used units of
analysis – country–year, dyad–year, and directed dyad–year. In addition, the software
enables the researcher to combine many of the events datasets discussed in this essay
with basic demographic and geopolitical data including data uploaded by the user.
Kansas Event Data System. At http:/web.ku.edu/keds/, accessed July 2009. KEDS

provides a computer program that enables users to specify and create personalized
events datasets. Detailed descriptions of machine coding methods, as well as several
datasets and codebooks, are available on the KEDS web page.
The Consortium on Qualitative Research Methods. At www.maxwell.syr.edu/moynihan/

programs/cqrm/index.html, accessed July 2009. CQRM promotes qualitative research
methods in the social sciences. They hold an annual training institute that may be useful
for FPA scholars interested in expanding their knowledge of qualitative methods. In
addition, they maintain a database of syllabae on qualitative methods.
Intrade. At www.intrade.com/, accessed July 2009. Intrade is the leading prediction

market for political futures. The site has prediction markets for a wide range of political
and financial outcomes that may be of interest to FPA scholars.
Page 28 of 29


Acknowledgments
I would like to acknowledge the helpful comments of Michael Glosny, Deborah Larson,
Rachel Augustine Potter, and two anonymous reviewers. All errors are my own.
Philip B.K. Potter
Woodrow Wilson Department of Politics, University of Virginia
Page 29 of 29


Potter Methods of Foreign Policy Analysis

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Potter Methods of Foreign Policy Analysis

Transféré par

Droits d'auteur :

Formats disponibles

Methods of Foreign Policy Analysis

Oxford Research Encyclopedia of International

Summary and Keywords

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

The Methodological Origins of Foreign Policy

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

Simultaneously, in response to the near monopoly of system-level theory in international

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

the generalizability garnered by operationalizing foreign policy interactions numerically

Comparative Foreign Policy and Events Data Analysis

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

The World Event/Interaction Survey (WEIS)

The Conflict and Peace Data Bank (COPDAB)

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

International Crisis Behavior Project (ICB)

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

The ICB project is unusual in that it proceeds simultaneously at multiple complementary

Correlates of War Project (COW)

Trade Dependence and Foreign Policy Compliance

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

Events Data – Methodological Challenges

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

The non-independence of foreign policy events presents an additional methodological

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

Qualitative Methods of Foreign Policy Analysis

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

It is admittedly artificial to divide methods of foreign policy analysis by “generation,” as

Case Study Analysis

There is no shortage of examples of the excellent use of case study methodology in

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

An example of such collaboration can be found in a relatively recent volume edited by

Gathering Qualitative Data

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

There are a number of excellent examples of innovative interview methods in foreign

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

Third Generation Methods of Foreign Policy

Computer assisted coding of electronically stored information offers several advantages

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

Experiments and Simulation in Foreign Policy Analysis

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

This leap can be particularly worrisome when it is from an experimental finding

The International Communication and Negotiation Simulations (ICONS) project is an

Survey Research in Foreign Policy Analysis

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, INTERNATIONAL STUDIES (oxfordre.com/internationalstudies).

Subscriber: Universidad de Chile; date: 06 March 2019