Evaluation and Program Planning 35 (2012) 354369

Contents lists available at SciVerse ScienceDirect

Evaluation and Program Planning

journal homepage: www.elsevier.com/locate/evalprogplan

Evaluating theory-based evaluation: Information, norms, and adherence

W. Jake Jacobs a,*, Melissa Sisco a, Dawn Hill a, Frederic Malter a,b, Aurelio Jose Figueredo a
a b

Department of Psychology, 1503 E University Blvd., PO Box 210068, Psychology Bldg. Rm. 312, Tucson, AZ 85721, United States Evaluation, Research, and Development Unit (ERDU), 2030 E. Speedway Blvd. Tucson, AZ 85719, United States

A R T I C L E I N F O

A B S T R A C T

Article history: Received 5 May 2010 Received in revised form 23 October 2011 Accepted 4 December 2011 Available online 13 December 2011 Keywords: Program evaluation Experimenting society Psycho-educational intervention Social norms Norm adherence Instrumental verbiage Instrumental behavior Rule Governance Altruistic punishment Automaticity Declarative routines Procedural routines Smoking prevention programs Pro-environmental programs Sexual violence prevention programs

Programmatic social interventions attempt to produce appropriate social-norm-guided behavior in an open environment. A marriage of applicable psychological theory, appropriate program evaluation theory, and outcome of evaluations of specic social interventions assures the acquisition of cumulative theory and the production of successful social interventions the marriage permits us to advance knowledge by making use of both success and failures. We briey review well-established principles within the eld of program evaluation, well-established processes involved in changing social norms and social-norm adherence, the outcome of several program evaluations focusing on smoking prevention, pro-environmental behavior, and rape prevention and, using the principle of learning from our failures, examine why these programs often do not perform as expected. Finally, we discuss the promise of learning from our collective experiences to develop a cumulative science of program evaluation and to improve the performance of extant and future interventions. 2012 Published by Elsevier Ltd.

1. General introduction The central purpose of this paper is to illustrate how program evaluation can be used to generate cumulative scientic knowledge. Such knowledge may be useful to policy makers when developing social programs and to program evaluators when evaluating intervention programs containing certain ineffective common elements but omitting other more effective elements. To accomplish this, we review three distinct kinds of informationonly or psycho-educational interventions in the elds of rape prevention, pro-environmental behavior, and smoking prevention. We review the program theory underlying these information-only interventions and critique those interventions from the perspective of Pragmatic Behavioral theory. We then review the results of

* Corresponding author. Tel.: +1 520 626 4825; fax: +1 520 621 9306. E-mail addresses: wjj@u.arizona.edu (W.J. Jacobs), sisco@u.arizona.edu (M. Sisco), dawnh@email.arizona.edu (D. Hill), fredericmalter@WEB.DE (F. Malter), ajf@u.arizona.edu (A.J. Figueredo). 0149-7189/$ see front matter 2012 Published by Elsevier Ltd. doi:10.1016/j.evalprogplan.2011.12.002

several program evaluations in the three domains and compare the empirical results to theoretical predictions. We begin by discussing the promise for this kind of programtheoretical review and analysis for the prospect of creating cumulative evaluation theory. The basic point of the exercise is that, as long as we approach evaluations in a piecemeal program-by-program fashion, we cannot create a cumulative body of knowledge, and we will continue to implement ineffective social interventions. On the other hand, by learning from our collective experiences and developing a cumulative science of program evaluation, we can implement effective methods that improve the performance of extant and future interventions. Fortunately, there have been great strides made in recent years in accomplishing that goal in several areas of evaluation research. Our immediate goal is therefore to apply these promising approaches to the problem of evaluating psycho-educational interventions. To accomplish these objectives, we need to match program evaluation theory and applicable psychological theories governing mechanisms that produce open environment social-norm-guided behaviors targeted by psycho-educational interventions.

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

355

We therefore begin this integration by describing a need to develop cumulative theory in program evaluation, based on relevant principles developed within the eld of program evaluation, and citing contemporary examples where this ongoing endeavor meets with provisional success. Based on these principles, we then reconstruct and articulate a general theory, based on some well-known examples from environmental education that we assert underpins virtually all psycho-educational programs. We then propose an alternative model that describes several properties and operating characteristics of wellestablished learning mechanisms and psychological processes involved in changing social norms and social-norm adherence targeted by many psycho-educational interventions. Our immediate goals are therefore not to critique or somehow redirect program evaluation but to merely practice it ourselves, as numerous program evaluation theorists cited below strongly recommend, by gleaning cumulative knowledge regarding the performance (or lack thereof) of psycho-educational interventions across several domains, and proposing a body of relevant behavioral theory to account for these results. For the sake of brevity within the present work, we refer the interested reader to a book chapter in the Oxford Handbook of Quantitative Methods, containing numerous critiques and recommendations for the practice of program evaluation, now in press, for a fuller treatment of that subject matter (Figueredo, Wolf, Olderbak, Schlomer, & Garcia, in press). For now, our immediate intent is not so much to draw implications for program evaluation, as to draw implications from program evaluation for present and future public policy. In so doing, however, we outline a heuristic model that other program evaluations might nd useful in the evaluation of psychoeducational interventions.

cases and will be difcult to do. The quest will probably be more appropriate for meta-analysis than for single studies. (p. 52) This theory-testing enterprise presupposes an experimenting society that derives cumulative knowledge from a history of social programs and principled evaluation outcomes rather than from a history of program-by-program trial-and-error learning (Campbell, 1971/1991; Lipsey, 1993). Similarly, Brownson, Gurney, and Land (1999) proposed that accumulating this type of evidence leads to recommendations for more specic programs of action, and that . . . the impetus for action is strengthened by consistent ndings from a series of well-conducted studies (p. 90). A serious problem impeding the creation of program theory, however, is that those designing or implementing the program often do not articulate such theory a priori. Leeuw (2003) described methods for reconstructing often imprecise and only implicit theories underlying programs and policies, articulating them a posteriori. This imprecision frequently necessitates an inductive hunt for the causes implicitly assumed to produce intended changes. Because this process of program theory reconstruction is fallible, evaluators may incorrectly reconstruct the basal theories guiding a program. Leeuws paper therefore focuses on several alternative methods for reconstructing and articulating implicit program theory. Despite these problems: . . . there are three possible responses to the challenge of causal attribution. One is to give up the attempt to use program theory evaluation for this purpose, deciding to use it only to improve, not to prove. Another option is to combine program theory with other methods for causal attribution . . .. Alternatively, a Popperian approach can be taken, and program theory can be used to develop testable hypotheses, which are then investigated using nonexperimental methods. Using the variations among different levels of implementation and different contexts for implementation not as noise to be screened out but rather as opportunities to test hypotheses, one can build a stronger case that the program not only contributes to the observed outcomes but also to explaining how. (Rogers, 2007, pp. 5556) Indeed, there are now at least some ongoing major metaanalytic initiatives directed towards accomplishing these worthy, longer-term, and ambitious objectives. For example, The Campbell Collaboration (www.campbellcollaboration.org) is an international research network that produces systematic reviews of the effects of social interventions, based on voluntary cooperation among a diverse array of researchers from different elds. Then there is the What Works Clearinghouse (http://ies.ed.gov/ncee/wwc/), established as an initiative of the U.S. Department of Educations Institute of Education Sciences, which serves as a central source of scientic evidence for what works (and presumably what doesnt work) in education. The WWC lists their major services as follows: Produces user-friendly practice guides for educators that address instructional challenges with research-based recommendations for schools and classrooms; Assesses the rigor of research evidence on the effectiveness of interventions (programs, products, practices, and policies), giving educators the tools to make informed decisions; Develops and implements standards for reviewing and synthesizing education research; and Provides a public and easily accessible registry of education evaluation researchers to assist schools, school districts, and program developers with designing and carrying out rigorous evaluations.

2. Cumulative theory in program evaluation Campbells (1971/1991) famous paper, Methods for the Experimenting Society, outlined how society can use the results from program evaluations to increase its knowledge of the kinds of social interventions that do and do not work. Campbell envisioned a perpetually experimenting society that uses its collective experiences to assess the results of different social programs, revises, and improves those programs in light of the results of previous assessments. The outcome of formal program evaluation provided critical feedback based on those results. Lipsey (1993) elaborated this basic idea, proposing that we learn the most from program evaluations that test specic causal theories of process mediation rather than treat programs as black boxes: Treatment theory is a set of propositions regarding what goes on in the black box during the transformation of input to output (p. 11). According to Weiss (1998), program theory (for present purposes synonymous with treatment theory) . . . refers to the mechanisms that mediate between delivery (and receipt of the program and the emergence of outcomes of interest (p. 57). Similarly, Rogers, Petrosino, Huebner, and Hacsis (2000) Program Theory Evaluation (PTE), . . . consists of an explicit theory or model of how the program causes the intended or observed outcomes and an evaluation that is at least partly guided by this model (p. 5). Weiss (1997), however, noted that testing treatment theory is not trivial: Another example would be efforts to change such behaviors as low school grades, delinquency, and domestic violence through programs that seek to raise self-esteem and self-condence. Theory-based evaluation could be directed at investigating the viability of such central theoretical premises. Evaluations that test such macro-theoretical assumptions will require multiple

356

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

We therefore begin our review with a reconstruction of the program theory presumably underlying psycho-educational interventions, then articulate a functional appraisal of psychosocial mechanisms governing adherence to social norms and compare the basic assumptions of the general theory presumably underlying psycho-educational programs to these fundamental principles. 3. A reconstructed general theory underpinning psychoeducational programs We begin our reconstruction of the underlying program theory using Environmental Education (EE) as an example of a representative psycho-educational intervention. In 1975, the United Nations adopted the Belgrade Charter as a goal statement for environmental education. According to this document, the single ultimate goal of EE was as follows: To develop a world population that is aware of, and concerned about, the environment and its associated problems, and which has the knowledge, skills, attitudes, motivations and commitment to work individually and collectively toward solutions of current problems and the prevention of new ones (UNESCO, 1975). In 1977, the Intergovernmental Conference on Environmental Education (organized by UNESCO in cooperation with UNEP) subsequently adopted the Tbilisi Declaration, which, built upon these goals, provides the foundation for much of what has been done in EE since that time. This document updated the ultimate goals of EE as follows: To foster clear awareness of and concern about economic, social, political, and ecological interdependence in urban and rural areas; To provide every person with opportunities to acquire the knowledge, values, attitudes, commitment, and skills needed to protect and improve the environment; To create new patterns of behavior of individuals, groups, and society as a whole towards the environment (UNESCO, 1978). The third point in the Tbilisi Declaration is worthy of note, in that it goes beyond the purely internal and psychological states of awareness, knowledge, values, attitudes, and skills as objects of change and, instead, identies the creation of new patterns of behavior as a goal for EE. Thus, according to both the Belgrade Charter and the Tbilisi Declaration, the specic objectives of EE included helping individuals and social groups acquire pro-environmental awareness, knowledge, values, attitudes, and skills, dened in nearly identical terms. The Belgrade Charter also included evaluation ability, which was subsequently removed from the Tbilisi Declaration, for unexplained reasons. The last item on this list of objectives, participation, contained some interesting differences in wording between the two successive documents. As the nal objective of EE, the Belgrade Charter dened participation as: . . . to help individuals and social groups develop a sense of responsibility and urgency regarding environmental problems to ensure appropriate action to solve those problems (UNESCO, 1975). In contrast, the Tbilisi Declaration dened participation as: . . . to provide social groups and individuals with an opportunity to be actively involved at all levels in working toward resolution of environmental problems (UNESCO, 1978).

What is most interesting about this difference in phrasing is that the rst document targets a sense of responsibility and urgency as putatively causal to appropriate action, whereas the second targets an opportunity to be actively involved as a potential contribution to working towards the resolution of environmental problems. Nevertheless, it is evident that both documents credit internal psychological states, such as awareness, knowledge, values, and attitudes, as necessary and possibly sufcient for promoting changes in behavior. In our view, this reasoning is a representative example of the general theory we argue underpins virtually all psycho-educational programs. The idea is that the information provided by an educational intervention will somehow alter internal psychological states, and that changes in these internal cognitive and affective states will translate into changes in overt behavior. 4. The ABCs of social norms and norm adherence Most evaluation researchers assume that social norms play a key role in guiding pro- and anti-social behavior (e.g., Krug, Mercy, Dahlberg, & Zwi, 2002). Such behavior includes those related to intimate partner violence, health, the environment, voting, drug and alcohol use to name a few. Moreover, a number of studies conducted over the past 50 years or so demonstrates that attitudes, taken in the context of subjective norms, predict intent to act and that intentions are good predictors of behavior (see Fishbein & Ajzen, 2010 for a review). Some programs, based on carefully articulated lab-based principles, seem to work, but some purely information-based programs may boomerang (e.g., Fishbein, HallJamieson, Zimmer, von Haeften, & Nabi, 2002). Nevertheless, many claim that a key to changing behavior is to change social norms. For example, a report dated 2000 from the US Department of Health and Human Services concluded: . . . comprehensive approaches combining community interventions, mass media campaigns, and program policy and regulation are most effective in changing social norms and reducing tobacco use. While acknowledging the notion that changing social norms may affect behavioral change, many, if not most information-based intervention programs target changes in personally held normative rules guiding the lives of these individuals rather than socially held normative rules guiding the lives of most of us. These programs often target the individual, without considering the various contexts within which these individuals live. We assert these contexts, consisting of socially held norms, often at variance with broader social norms, matter. There are two major types of social norms: (1) Descriptive Norms, which are statements of fact regarding what is typical within a society, and (2) Injunctive Norms, which are normative rules or beliefs regarding what constitutes morally approved or disapproved conduct. In philosophical terms, descriptive norms specify what actually is, whereas injunctive (prescriptive) norms instead specify what ought to be (Cialdini, Reno, & Kallgren, 1990; Deutsch & Gerard, 1955; Schaffer, 1983). In the ensuing discussion, we limit ourselves to the role of injunctive social norms in psychoeducational interventions and behavioral change. We shall assume that a tendency to give and follow social norms guides our moment-by-moment interaction between the physical environment and us (Watch out for low-hanging branches!) and among ourselves (Stop on red, go on green, exert caution on yellow). Such norm giving and following keeps us civil, permits civilized cooperation, and lets us reap the benets of the experience of others without the costs of trial-and-error learning.

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

357

We also assume that social norms, and the fact that humans in general follow them, not only guide personal interactions, they form the fabric of our societies. Although what is normative varies among cultures, cooperation within human groups (group cohesion) depends upon following appropriate often poorly stated but reasonably well understood social norms and moral principles. These facts lead us to several questions. Among those questions, the most important are, What are injunctive social norms?, What is the nature of adherence to injunctive social norms?, and How can appropriate social programs promote adherence to injunctive social norms? Social norms, as used in the present manuscript, are a product of actions (e.g., verbiage/gestures, signs/symbols/symptoms) that functionally guide behavior. We can often describe or transmit norms, dened in this way, with a verbal statement in the form of a syllogism: if A and if B then C which specify or point to contingent relations among Antecedent conditions, target Behaviors, and probabilistic Consequences of the target Behavior.1 A norm may occur as a single if if then statement or as a set of such statements. A norm may be explicit or implicit and, because it is a part of a verbal community, written, spoken, or signed. Importantly, these statements may specify social norms to various degrees, ranging from partial to full specication (see the Glossary for theoretically driven denitions of these terms). Take, for example, a recent interaction between the senior author (WJ) and his then 5-year old son (Adam). WJ came home from work to nd the living room strewn with toys and Adam sitting on a chair watching television. WJ said, Son, please pick up these toys. Adam replied, Just a minute Daddy, let me nish watching this show. WJ replied, Adam, please pick up these toys or I will throw them out. Adam replied, Okay Daddy, turned on the TiVo (a device that records television programs), picked up the toys, and put them away in his room. We will use this example for other purposes later in the manuscript, but for now focus on WJs rst statement. It is an example of a partially specied culturally bound normative rule: keep clean. Unpacked, the normative rule applied to this case reads, if Daddy is home, grumpy, and says, Son, please pick up these toys (A1), toys are strewn about the living room (A2), and if you (Adam) adhere to the rule (B1) and pick up the toys (B2), then no aversive consequences will ensue (C1) and the toys will be picked up (C2).2 Although Adam required WJ to unpack the normative rule more completely, Adams behavior came under the inuence of the instantiated norm; he placed his toys in the toy box, and returned to his show. There was a small personal cost to Adam, time away from an interesting show, but the cost was far less that the perceived consequences of adherence failure. 4.1. Empirical demonstrations A long-standing experimental literature provides a number of laboratory-based demonstrations of the phenomena we just described. Under some circumstances, adult humans blindly follow verbally stated social norms even to their own personal detriment (e.g., Galizio, 1979; Hayes, Brownstein, Zettle, Rosenfarb, & Korn, 1986; Lippman & Meyer, 1967; Hayes, 1993; Kaufman, Baron, & Kopp, 1966). A number of models describe
1 We may think of these rules within the connes of conditional logic which permits linking many if statement through set intersection. 2 Notice there are two sets of ABC contingencies here. The rst A1 B1 C1 concerns conditional relations among rule (the norm itself), the behavior (adherence) and the consequent (what happens if adherence fails. The second A2 B2 C2 concerns conditional relations among the state of the room (strewn toys), the specied behavior (pick up the toys), and the consequences (the room will be clean and Daddy will be off of your back). See the Adherence as action section below for a more complete explanation.

the phenomenon (e.g., Doll, Jacobs, Sanfey, & Frank, 2009; Miller & Cohen, 2001). Kaufman et al. (1966), for example, brought 20 undergraduates to their laboratory individually, asked each to give up their watches, pencils, pens, books and gave one of ve sets of instructions before the undergraduates entered an actual experimental task. Although Kaufman et al. (1966) ran ve groups, only three are of interest here. The individuals in these groups received common instructions followed by instructions specic to the group. Those in a Congruent Variable Interval group received accurate instructions claiming that, once in the experimental chamber, pressing a choice button would payout every minute on the average. This instruction implied that the most efcient response strategy was to press the button slowly and steadily because, although the timing of the payout was unpredictable, it would occur, contingent upon a certain behavior, with the mere passage of time. Those in an Incongruent Variable Ratio group received inaccurate instructions claiming that, once in the experimental chamber, pressing a choice button would payout only after the undergraduate pressed the button 150 times on the average. This instruction implied that the most efcient response strategy was to press the button rapidly and steadily because, although the timing of the payout was unpredictable; how often it happened depended on how many button presses occurred. Those in an Incongruent Fixed Interval group received inaccurate instructions claiming that responses on a choice button would payout once and only once every 60 s, no matter how often the subject pressed the choice button. This set of instructions implied that the most efcient response strategy was to wait for 60 s and then press the button once for a payout. After receiving these instructions, all of these undergraduates entered an experimental chamber and pressed a choice button that illuminated a green choice light on a Variable Interval 1-min schedule. Once the green light was illuminated, the undergraduate pressed one of two buttons immediately below it, which produced points indicating a correct choice on an increasingly dense probability schedule. Pressing either of the two buttons turned the green choice light off and reinstated illumination of the choice button. The undergraduate then pressed the choice button until the green choice light illuminated, pressed one of the two buttons immediately below it, received points according to the predetermined schedule, and the procedure recycled. The undergraduates repeated this cycle for 3-h and then exchanged the points for monetary compensation. Those given an accurate normative rule (the Congruent Variable Interval group) pressed the key as predicted, immediately and at a steady and constant rate. Those given an inaccurate normative rule pressed the key in patterns appropriate to the inaccurate normative rule rather than the actual payoff schedule. That is, when told that the payoff was on a variable ratio schedule (the Incongruent Variable Ratio group), the people pressed the key at high and constant rates. When told the payoff schedule was on a xed interval schedule (the Incongruent Fixed Interval group), the people pressed slowly shortly after receiving payoff and more quickly as the interval progressed even though everyone was on exactly the same payoff schedule. As this example illustrates, normative rules can and, under some circumstances, do override obvious environmental contingencies, a relation documented many times (e.g., Barush, Kanter, Busch, Richardson, & Barnes-Holmes, 2007; Catania, Shimoff, & Matthews, 1989; Galizio, 1979; Joyce & Chase, 1990; Hackenberg & Joker, 1994; Hojo, 2002; LeFrancois, Chase, & Joyce, 1988; Lippman & Meyer, 1967; OHora, Barnes-Holmes, Roche, & Smeets, 2004; Smeets, Dymond, & Barnes-Homes, 2000). To understand, at least partially, conditions under which such normative rules govern behavior, we briey turn to a

358

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

well-established view of the inuence of context and consequences on behavioral organization, and then return to the example. Traditionally, those interested in descriptive models of performance accept that a three-term contingency lies at its core. This set consists of Antecedent conditions (A), target Behavior (B), and behavioral Consequences (C) hence, an ABC view of behavioral organization. During recent years, methodological behaviorists (including many cognitive psychologists, cognitive neuroscientists, and some linguists) focused on the mechanics of Antecedent conditions (e.g., central processes such as Attention, Executive Function, various forms of Memory and the like) as descriptors that may help us understand the conditions immediately antecedent to any given behavioral class. Those interested in norm adherence, however, focused on relations between normative statements (rule specication) and the consequences of adherence failure. Those interested in norm adherence tell us that a norm species a set of Antecedent conditions (if these conditions obtain), a set of Behavioral strategies (and if you do this), a Consequent (then that will or will not happen), and conditional relations among the Antecedent, Behavior, and Consequences to various degrees of specicity. They also tell us that, as a descriptive model, it is not necessary to specify the internal causal dynamics involved in each if or then statement (e.g., that the rst if changes the brain in such a way that particular cognitive resource become active whereas others become inactive). Instead, it is enough to observe, record, and report systematic relations among each of these variables (A, B, and C). As described above, it is an empirical fact that, under certain conditions, stating a normative rule dramatically reorganizes the relations among observable A, B, and C (e.g., Doll et al., 2009). The interaction between Adam and WJ illustrated this point as do many of the more dramatic examples provided by the social psychologists (e.g., Milgram, 1963; Sherif, Harvey, White, Hood, & Sherif, 1961; Zimbardo, 2006, 2007) and others (e.g., Bloom, 2005). The example of the interaction between Adam and WJ, however, offers a unique point; norm adherence appears to be a malleable (docile) behavior or action (e.g., Cerutti, 1989). 4.1.1. Adherence as action Thus far, we have used the theoretical construct behavior intuitively. If, however, we are to use behavior as a theoretical construct, it is best to have at least a rough idea of what that construct is, thereby avoiding confusion and unnecessary debate. For folk psychology, the term behavior implies little more than directly observable (in the moment) activities of organisms. For those trained in (Pragmatic/Functional) behavioral psychology, however, the construct behavior implies much more and much less (e.g., Jacobs et al., 1988; Pfaus et al., 1988). We begin with more. The technical meanings of the construct behavior come in at least two avors (e.g., Kantor, 1970). The rst, organismic behavior, references variables or factors observable mainly in the movements or acts of organisms, without considering contextual features. This is similar to the folk psychological conception of behavior.3 The second construct, behavioral elds (action), includes constraints imposed by setting factors (that is, affordances, enabling and impeding organismic responses) and the impact of organismic behavior (not the organismic behavior itself) on associated environmental objects (e.g., Kantor, 1970; Jacobs et al., 1988).4 It is in the second sense that we use the construct behavior, as a class of events dened by function, in the present context.

Now consider the less. Behavior, as used by folk psychology, for example, involves (at least for vertebrates) a complex causal, observable, and measureable interplay among a large number of muscles and interconnected bones (e.g., Eshkol & Wachman, 1958; Jacobs et al., 1988). Each joint represents a specic number of degrees of freedom that, when added, yield a complex system comprised of a large number of movement (e.g., Benjamini et al., 2010; Golani, 1976; Pellis, 2010; Teitelbaum et al., 2004). In contrast, behavior, as used by those trained in pragmatic behavioral psychology, ignores specic organismic or phenotypic behavior (to a large part) focusing instead on a description of an interplay among setting/situation variables (Antecedent conditions), a large (and indenite) set of organismic behaviors, and the impact on setting or situational variables brought about by that set of organismic behaviors (Consequences).5 In conceiving of behavior in this way, behavioral psychology accomplishes two scientic goals. First, the eld achieves a tractable behavioral taxonomy. Rather than dealing with the degrees of freedom involved in organismic behaviors, the taxonomic rules permit us to reduce behavior to general functional characteristics (e.g., Type I versus Type II behavior, Konorski, 1967; operant versus respondent, Skinner, 1938). Second, in so doing, the term behavior refers to a construct (a latent variable) that one may measure indirectly through an aggregate of its characteristics. The use of behavior to refer to a construct rather than a set of organismic variables changes the meaning of the term radically. Instead of referring to specic organismic behaviors, the theoretical meaning of behavior ignores the specics of the body and what it does instead the construct refers to and is dened by a description of functional relations among environmental (setting and situational affordances), potential adaptive problems in that environment, and the environmental changes brought about by any organismic behavior (e.g., Figueredo et al., 2007). Hence, the modern Functional Analyst may treat e.g., learning, attending, remembering, hoping, kindness, aversion, fear, courage, or despair as examples of a class of behavior a latent construct more commonly known as actions which are appropriate for rigorous ABC analyses. Notice here, although an observer cannot see or measure actions such as seeing, attending, or remembering directly even common folk language captures the notion directly as an act of seeing, an act of attending or an act of remembering. Hence, an observer must infer the existence of the latent construct (seeing, attending, remembering) based on relations between Antecedents and Consequences presumably caused by an indenitely large set of organismic behaviors (acts), that are nonetheless directed towards a common function (action). In more formal terms, the construct behavior (an act) lacks a denite behavioral (in the folk psychological sense) referent. In short, psychologists who depend upon behavior for their raw data, such as psychologists who study sensation and perception, cognition, social, personality psychology to name a few must, measure samples of pre-specied, behavioral characteristics that is, characteristics of the latent variable labeled behavior or action. Theoretically, these characteristics are caused by changes in a setting, situation, (e.g., Jacobs et al., 1988) brought about by organismic behavior. These psychologists, based on representative measures of the full suite of behavioral characteristics, then infer the strength, probability, or status of the latent variable (behavior) in question (e.g., seeing, attending, remembering). Hence, methodological behaviorists typically study measurable characteristics of behavior. That is, methodological behaviorists study the dynamics of a Latent Construct a construct that the
5

3 4

One might label this phenotypic behavior. One might label this instrumental behavior.

This is sometime known as a functional denition of behavior.

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

359

researcher cannot observe directly, but whose characteristics the researcher infers from the dynamics of a sample of indicators (e.g., facial expression, questionnaire answers, reaction time, search path taken in a virtual environment). That is, the researcher infers the dynamics of a Latent Construct through the behavior of a set of observable and measurable characteristics presumably caused by that Latent Construct. All of this leads us to the point of this section the status of adherence as a theoretically legitimate example of a Latent Construct which we may label behavior. Common language recognizes, at least obliquely, the status of abstract behavioral constructs. Take, for example, the phrases an act of kindness, an act of aggression, an intentional act, or an obedient or disobedient act. An act of kindness is an action that may entail a host of organismic behaviors; but is in no way dened by those organismic behaviors. Instead, the interplay between Antecedent conditions and Consequences brought about by a set of organismic behaviors dene an act of kindness (the specic organismic behavior is irrelevant to the construct). So too it is with an act of adherence. As with all other Latent Variables, measuring adherence to a rule as an action involves both setting and situation, and involves (at least initially) a specic aspect of the situation observable rules imbedded in and a part of the situation. 4.1.2. Acquiring adherence If we consider adherence to normative rules (governance) as a malleable (docile) action, we might ask, How does one inuence governance? To help answer that question we return to the WJ and Adam example. Although the if these Antecedents and if this target Behavior as well as the relation between them were evident in his statement, WJ did not clearly specify the consequences of a governance failure. Adam, who lacked experience with the full set of governance contingencies, declined to cooperate (he exhibited weak to non-existent governance). This led to a second statement, Adam, please pick up these toys or I will throw them out that specied the Antecedents (the here and now), the target Behavior (pick up the toys), and the consequent of governance failure (Ill throw them out). Obviously, the strength of the instrumental behavior we label governance changed as a function of the unpacked statement.6 We consider this set of contingencies in the next section. 4.2. Achieving norm governance The general purpose of a social norm, which can be stated verbally as if this Antecedent and if this target Behavior, then these Consequences, is to bring an individual into contact with extant social consequences produced by social norm non-adherence. To simplify matters, these contingencies, which consist of relations among behavior and its consequences in the presence of Antecedent conditions, may fall into two general classes categorized by the temporal relations between the behavior and its consequences. 4.2.1. Immediate consequences The rst category, immediate Consequences, refers to a condition in which the target Behavior produces immediate
6 This example contains another fact that we can consider only briey. Adam discriminates Antecedent conditions quite well. When his father, who consistently acts on specied consequences, states a normative rule, norm adherence is relatively strong. That is, Adam adheres to the normative rule consistently. When his mother, who less consistently acts on specied consequences, states a normative rule, norm adherence is relatively weak. That is, Adam sometimes cooperates and sometimes does not (Jacobs et al., in preparation provide three experimentally based demonstrations of contextualized governance of behavior by social norms).

Consequences that stabilize (positive consequences) and/or destabilize (no or negative consequences) the target Behavior. This is well understood behavior stabilizes (e.g., increases in probability) or destabilizes (e.g., increases in variability) as a function of its consequences. In this case, a stated social norm acts in a way that alters the base rate of the target Behavior in the presence of specied Antecedent conditions. Hence, the target Behavior can come into immediate contact with effective Consequences and is maintained or enhanced. That is, stating a normative rule may bring target behavior under the inuence of extant BehaviorConsequent contingencies straight away.7 Notice that the major inuence of the norm statement on the target Behavior is to change the base rate of the target Behavior. Extant contingencies, related to the Consequences of the target Behavior, then directly inuence and maintain that behavior. Put bluntly, the inuence of the stated norm disappears. Theoretically, the norm statement is not stored, represented, remembered, or retrieved. With its function completed, the norm and its associated adherence drops out of the repertoire.8 4.2.2. Delayed consequences The second category, Delayed Consequences, refers to a condition in which the target Behavior produces delayed (at times on the order of days, months, years, or even decades) consequences. A classic example is that contained in the normative rule Dont Smoke. To unpack the rule: if, during your lifetime (Antecedent), you smoke regularly (target Behavior), then you have a high probability of developing lung cancer or a cardiovascular disease and dying prematurely (Consequences). Here, the analysis regarding strong governance (adherence) and the recommendations based on the analysis becomes somewhat more complex. As before, the ultimate role of the normative rule is to bring the target Behavior in contact with extant environmental contingencies. As before, the proximate role of the normative rule is to alter the base rate of target Behavior in the presence of specied Antecedent conditions. But, because the Consequences of the target Behavior are too delayed to stabilize the target Behavior, the theory predicts the target Behavior will destabilize. That is, the theory predicts increases in the variability of the target Behavior, which will thereby decrease the probability of the target Behavior re-occurring. This creates a bit of a puzzle. We observe, under some circumstances, stabilization of target Behavior under the inuence of a normative rule specifying temporally delayed consequences when this happens, the question becomes, What stabilized the behavior? Conceptually, stabilization of target Behavior has to do with interplay among the individual(s) who delivers a norm, the individual whose target Behavior is brought under the governance of the norm, and the individual(s) who enforce(s) the norm. Consider the simple proximate rule, if you clean the backyard, then I will give you 100 dollars. The probability of the target Behavior changes and stabilizes it appears to contact the 100dollar contingency. The probability of the target Behavioral chain (i.e., whatever organismic behaviors it takes to get the back yard clean) remains high and stable. Although the target Behavior is apparently under the inuence of the promised 100 dollars, it is not. It is instead under the inuence of the behavior of the individual who promised the 100 dollars, particularly how consistently he (or his kind) has delivered on specied consequences (particularly adherence to stated rules) in the past.

7 8

We might call this no trial or zero-trial learning. Automaticity develops see below for a more complete description.

360

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

4.2.3. Altruistic punishment The analysis moves forward in this way. Although normative rules inuence the base rate of target Behavior, the presence of the rule, coupled with an experiential history of oft unspoken contingencies surrounding such a rule, ensures a high and stable base rate of the target Behavior. Put another way, the contingencies surrounding the normative rule statement ensures that governance (as an action) remains high and stable. Reect, for a moment, on a fact we described earlier: Adherence to a normative rule is a malleable action sensitive to its consequences. If that is true, then theoretically, there are contingent relations supporting or undermining adherence. That is, there are identiable relations among a normative rule statement, a set of Antecedent conditions, adherence (a functionally dened target Behavior), and a set of Consequences. In the case we used to illustrate the acquisition of normative rule adherence (Adam and WJ), and in the clean the backyard case we are using now, we pointed out that adherence failures produce aversive consequences a penalty (a discarded toy or a failure to obtain 100 dollars). As it turns out, in human interactions, adherence (or governance) failures often produce aversive consequences. Humans easily detect non-adhering (rule-breaking) conspecics (cheaters; see e.g., Trivers, 1971). Moreover, humans appear to be governed by both informal and formal injunctive rules of collective consequences (e.g., social norms, rules, and moral principles) that we apply to cheaters. People are generally intolerant of cheaters tending to perceive such people as aversive leading the cheated to adopt similar uncooperative strategies (tit-for-tat), to remove themselves from the interaction (defection), or, more commonly, administer retribution, thereby dispensing Altruistic Punishment to the rule breaker. The weight of the evidence suggests that people freely incur great personal cost to punish cheaters those freeloaders who do not pull their weight in cooperative rule-bound endeavors a nding that holds in every culture studied (e.g., Herrmann, Thoni, & Gachter, 2008). As a result, those who do not adhere to normative rules risk suffering for their malfeasance (see e.g., Carpenter, Bowles, Gintis, & Hwange, 2009; de Quervain et al., 2004; Egas & Riedl, 2008; Fehr & Gaechter, 2002; Herrmann et al., 2008; Ohtsuki, Iwasa, & Nowak, 2009). Obviously, administering punishment, even altruistically, can come at great personal cost to the altruist (Dreber, Rand, Fudenberg, & Nowak, 2008; Rand, Ohtsuki, & Nowak, 2009). Here, the rules governing the visitation of consequences on normative rule breakers primarily the costs that punishment visits on the altruist defy economic logic. Nevertheless, it appears that humans are quite willing to suffer personal cost to administer Altruistic Punishment and that norm adherence is sensitive to and inuenced by social sanctions applied systematically (and perhaps through biologically prepared principles) to those who break normative rules. 4.2.4. Adherence as an automatic avoidance response inuenced by understood sanctions All of this reminds us of a literature that developed between about 1930 and 1980 a literature that examined avoidance responding in other species. Consider a series of studies conducted by Richard Solomon and his colleagues using a standard active avoidance learning procedure. Here, the experimenter selected a response (a target Behavior) that the subject must emit to escape or avoid a painful aversive stimulus. The experimenter introduced the subjects to an experimental chamber, who then experienced three clearly dened conditions. The onset of a warning signal (an Antecedent condition) marked the beginning of a warning period; leaping over a barrier (the designated target Behavior) terminated the warning signal and initiated a safe period (a Consequence,

usually marked by the absence of the warning signal). If the subject did not emit the target Behavior during the warning period, a brief electric shock occurred repeatedly. The shock period continued until the subject emitted the target Behavior (an escape response), which produced a new safe period. Typically, avoidance learning occurred in several stages. Initially, the subject only responded during the shock period. Upon emitting the target Behavior (the escape response), shock and the danger signal terminated and the safe period began. As the subject encountered additional learning trials, the target Behavior occurred progressively earlier in the shock period. The subject then began to respond during the warning period, completely avoiding the shock period. With continued training, the subject responded only at the beginning of the warning period thus effectively avoiding shock (see e.g., Solomon & Wynne, 1953). Using this procedure with dogs, Solomon and his colleagues found that, once well established, an avoidance response is difcult to extinguish (Solomon, Kamin, & Wynne, 1953).9 Moreover, the avoidance response gained in strength as training progressed. These researchers also found that, with extended training, there was little or no behavioral or physiological evidence of fear exhibited during the avoidance sessions (see also, Gantt, 1953; Kamin, Brimer, & Black, 1963; Maier, 1949; Masserman, 1943; Solomon et al., 1953; Starr & Mineka, 1977). In short, during avoidance training, automaticity occurred avoidance responding became increasingly automatic over the course of such training. Recent neuroimaging literature makes it clear that Altruistic Punishment, whether administered as social rejection, approbation, monetary penalties, or the like produces neural responses similar to those produced by physical pain (e.g., Eisenberger & Lieberman, 2004; Eisenberger, Lieberman, & Williams, 2003; Immordino-Yang, McColla, Damasio, & Damasio, 2009) and that, for many of us, merely perceiving pain in other humans may cause similar responses (e.g., Jackson, Meltzoff, & Decety, 2005). As importantly, recent neuroimaging research makes it clear that performing an avoidance response activates neural structures and circuitry commonly associated with reward and reinforcement (e.g., Delgado, Rita, Jou, LeDoux, & Phelps, 2009; Kim, Shimojo, & ODoherty, 2006). Those facts, coupled with the conditions under which humans administer Altruistic Punishment, lead us to consider Adherence, at least in many cases, as an at least partially automated avoidance response. 4.2.5. Automaticity If we simultaneously examine the accepted characteristics of automaticity specied by more recent literature (e.g., Moors & De Houwer, 2006; Wood & Neal, 2007), and the characteristics of the acquisition and maintenance (stabilization) of avoidance responding (e.g., Solomon & Wynne, 1954; Mineka, 1979), a remarkable set of parallel features appear.10 If we think of an individuals experiential history as consisting of a multitude of training experiences, then we may begin to appreciate the contributions of that history to normative rule adherence. Very few of us consider which fork to use when dining out, acceptable language given the company we keep, where we deposit our bodily wastes, or whether or not we will work for a living. Much like Solomons dogs, we adhere to these rules automatically, guided by experiences that others used to teach us often using procedures relying on aversive consequences. Impor-

9 The behavior emitted by the experimenters (running the dogs) extinguished before the dogs behavior (avoidance) extinguished. 10 Here we ignore the models designed to explain the development of automaticity and, instead, focus on the features (characteristics) that dene it. Hence, we are ignoring explanatory (causal models) and using a taxonomic approach to the problem of classifying adherence properly.

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

361

tantly, we continue to adhere to these rules without the need of immediate or even a high risk of other-administered social sanctions the act of adherence itself appears to be rewarding. 4.2.6. Removing governance consequences A change in injunctive social norms will not affect governance unless two conditions obtain: the chance of being caught is high or governance (adherence) is specically trained as a context-free instrumental action. An anonymous reviewer approached the former assertion by wondering what might happen to governance, . . . if there were no reasonable chance of getting caught? Although it was tempting to describe recent or current events (e.g., the social consequences of the disruption Hurricane Katrina brought about, the ongoing looting because of police removal in Egypt, or even the anonymity effects so often observed by the social psychologists), it seemed more appropriate to answer this question with three direct examples from the experimental literature. We begin with an assertion: humans are not well-calibrated scientic instruments. Measures taken by humans are consistently inconsistent, unreliable, and subjective. We have known this at least since Wilhelm Bessel described signicant differences in observationally based measures taken by Navil Maskelyne and his assistant David Kinneybrooke (see Boring, 1950, pp. 134142 for a brief account). To increase the accuracy of human observers, researchers often train them to follow a system of well-articulated observational rules (e.g., Jacobs et al., 1988). These researchers are all too aware of the extraordinary and idiosyncratic variability in the ways humans see and record behavior and of the extraordinary persistence with which humans cling to their observational biases. It often requires months of training before a researcher can bring observers behavior under the governance of a set of clearly articulated objective observational rules. After extensive practice with relatively complex observational rules, observers typically agree about 70% of the time when they know others are checking their reliability; but agreement drops precipitously when we check those same observers without their knowledge (e.g., Kent, Kanowitz, OLeary, & Cheiken, 1977; Reid, 1970; Taplin & Reid, 1973). That is, adherence is relatively strong when the observer can be caught breaking observational rules but relatively weak when the observer cannot be caught breaking observational rules. Moreover, if an investigator informs observers of the expected observational outcome (his or her personal rules), observers tend to drift from adherence to the formal observational criteria (rules) towards adherence to the rules articulated by the investigator. Feedback from the principal investigator regarding the quality of the observational record intensies the effect (Kent, OLeary, Diament, & Dietz, 1974; Shuller & McNamara, 1976). Finally, over time and if unwatched, observers drift from the original observational rules. If alone and unwatched, observers drift in their own idiosyncratic ways; if in collaborative pairs and unwatched, observers drift together that is, inter-observer agreement remains high within the pair, but agreement with the original observational rules decays over time (DeMaster, Reid, & Twentyman, 1977; Kent et al., 1974; Romanczyk, Kent, Diament, & OLeary, 1973; Wildman, Erickson, & Kent, 1975). Data such as these indicate that stable rule adherence (governance), at least in the case of articially imposed rules, requires consequences. With consequences, governance appears to remain stable across time; without consequences, governance appears to decline precipitously. 4.2.7. Behavioral development Given the foregoing, we claim that rule governance (norm adherence) is a malleable behavior, initially under the control of

consequences (usually aversive), but that may becomes habitual or automatic. We do not claim that all governance (norm adherence) is experientially acquired, only that experience inuences governance, and that it remains plastic in response to changes in the environmental contingencies, although (as with language acquisition) the degree of plasticity may decrease with age or experience or both (e.g., Tees & Werker, 1984; Werker & Tees, 2005). The fact that most behaviors are partially genetic and partially environmental is a well-established principle in behavioral biology a principle demonstrated across a wide array of species and behaviors (West-Eberhard, 2003; Figueredo, Hammond, & McKiernan, 2006). In humans, most scientically studied behaviors constitute a complex mixture of nature and nurture. Although, natural and sexual selection might also help shape norm adherence over evolutionary time, behavioral biologists currently assert that any substantial behavioral evolution involving humans minimally requires the passage of millennia (Cochran & Harpending, 2009). Because this manuscript specically addresses the evaluation of psycho-educational intervention programs, which need to show effectiveness on a much shorter time scale, we limit ourselves to the consideration of the malleable components of norm adherence or rule governance, and leave inherited components to the behavioral geneticists. Furthermore, there are both context-specic and contextindependent components of rule governance. Adherence to a specic norm may be shaped by a context-specic contingency of reinforcement, following the ABC principle described above. A more general form of rule governance (norm adherence), however, may also be trained as context-free instrumental action. Altruistic punishment is an example of an apparently contextfree instrumental action. As described above, other humans routinely inict punishment upon cheaters for breaking any social norms, rather than limiting these consequences to the violation of one specic rule. In fact, evolutionary psychologists have presented evidence that humans are biologically prepared to enforce social rules of unknown or arbitrary content, meaning that they are quick to learn these rules and process them much more quickly and accurately than they do more abstract logical problems of the same general form (e.g., Cosmides & Tooby, 1992; Gigerenzer, Hertwig, & Pachur, 2011). If that is the case, then humans living in human societies will be exposed throughout development to more generalized contingencies of reinforcement where an overall pattern of rule-following behavior is systematically reinforced and an overall pattern of rulebreaking behavior is systematically punished, regardless of which particular rules are specically at issue at any single point in time. Thus, we claim that there should develop both context-specic and context-independent components of rule governance in humans, by virtue of the characteristics of the social environments which they typically inhabit. Although this pattern implies that there was genetic selection over evolutionary time for context-independent components of norm adherence, in addition to operant selection over developmental time, we limit ourselves to the latter processes for present purposes. 5. Word and deeds 5.1. Declarative and procedural targets of social norms Traditional cognitive psychology recognizes a distinction between declarative and procedural knowledge.11 The eld measures declarative knowledge as speaking about something
11 Researchers often call these declarative and procedural memories and operationalize both in the same way.

362

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

appropriately and procedural knowledge as doing something appropriately. The experimental literature surrounding this distinction clearly demonstrates that declarative and procedural behaviors depend upon separable and at least partially independent brain structures (see e.g., Eichenbaum & Cohen, 2004; Ullman, 2001 for reviews). This suggests that brain systems devoted to talking about something (instrumental verbiage) are at least partially independent of brain systems devoted to doing something (instrumental behavior) and leads to the fact that, although these two forms of behavior may occur simultaneously, each can be under the inuence of distinct, identiable sets of contingencies (see e.g., Baum, 2004). These facts lead us to our nal general point. The purpose of a psycho-educational program is to produce strong rule governance adherence to prescriptive norms through the action of Injunctive norms. The point is to bring the target (instrumental) Behavior of an individual or group of individuals from what it is (its current set of localized descriptive norms) to something else (the larger set of socially approved descriptive norms). Yet, psycho-educational programs seldom target instrumental behavior (procedural routines) of interest, instead they target, perhaps unintentionally, what people say about the topic of interest (instrumental verbiage or declarative routines). To put it another way, psycho-educational programs often directly teach people to speak about specic problems in appropriate ways; they seldom directly support people acting appropriately. To use the vernacular, the designs of many psycho-educational programs produce norms that permit people to talk the talk but not to walk the walk. We unpack this last point in the examples that follow. 5.1.1. Program successes and failures: KEEP and DARE To illustrate program and program evaluation successes and failures, Figueredo et al. (in press) used two representative case studies: one notable failure of the program evaluation process, Drug Abuse Resistance Education (DARE) and one notable success of the program evaluation process, the Kamehameha Early Education Project (KEEP). Although these authors focused mostly on exceptionally awed and exceptionally brilliant evaluation processes, respectively, we focus, instead, on the way the programs themselves were developed and implemented in light of the ABC principles we describe. 5.1.2. Drug Abuse Resistance Education (DARE) The essence of the DARE program is informational. Uniformed police ofcers deliver a psycho-educational curriculum in safe classroom environments aimed at preventing drug abuse among students. Hence, DARE is an intervention that provides little more than lectures to students on the dangers of drug abuse. By this classroom-based method, DARE purports to teach children skills need to resist peer pressures encouraging drug abuse or involvement in gang-related violence. According the Ofcial DARE Web Site (http://www.dare.com/home/about_dare.asp), instructional goals include: (1) humanizing the police so that the students relate to ofcers as people; (2) permitting students to see police ofcers in a helping rather than an exclusively law enforcement role; (3) opening lines of communication between law enforcement and students; (4) having police ofcers serve as sources of information beyond drug-related topics; and (5) opening dialogue between the school, police, and parents that might be benecial deal with other, meaning non-drug-related, concerns. This program entails providing classroom lectures on the assumption that information contained in these lectures will lead to novel patterns of social interaction between students and law enforcement personnel. Given the foregoing theoretical stance, it should be no surprise that evaluation research, spanning over two decades, demon-

strates DARE is, at best, ineffective and, at worst detrimental. One of the more recent meta-analyses estimated the average effect size for DAREs effectiveness was extremely small and not statistically signicant (r = 0.011; Cohens d = 0.023, 95% condence interval = 0.04, 0.08). In response to much of this research (e.g., Ennett, Tobler, Ringwalt, & Flewelling, 1994; Clayton, Cattarello, & Johnstone, 1996; Dukes, Ullman, & Stein, 1996), Satcher (2001, Chap. 5) placed the DARE. program in the Does Not Work category of programs and the U.S. General Accountability Ofce (2003) penned a letter to Congress citing a series of empirical studies in the 1990s which showed in some cases DARE is iatrogenic, meaning that DARE does more harm than good. Nevertheless, tax dollars through over a dozen major government agencies, including the U.S. Department of Defense, U.S. Department of Justice, Bureau of Justice Assistance (BJA), Drug Enforcement Administration (DEA), Ofce of Juvenile Justice and Delinquency Prevention, and U.S. Department of State continue to fund DARE (Figueredo et al., in press). The relevant point here is that the ABC principles we outline in the present manuscript not only provide an account of why this failure occurred, it provides predictions about changes one might implement to turn DARE around. 5.1.3. Kamehameha Early Education Project (KEEP) Tharp and Gallimore (1979) described a classic example of a successful program evaluation: the Kamehameha early education project (KEEP). The KEEP program began 1970. Its goals were to improve reading and general education of Hawaiian children. The research and development process for this program consisted of three stages. The rst stage established a baseline of descriptive knowledge regarding the actual behavior of Hawaiian children as it plays out in their native culture as well as in the school system. This was done primarily using naturalistic observational sampling of the behavior of Hawaiian children in the context of socialization by their families and communities, and then in the context of the school setting to which they are later exposed. There were marked contrasts between the observed school culture and the observed home culture, where culture was dened as patterns of behavioral activities and social interactions rather than abstract sets of attitudes, beliefs, or cognitions (Tharp & Gallimore, 1979; Tharp & Gallimore, 1982). The second stage of program development created an effective educational program within a laboratory school. This experimental program explicitly made the work contexts and the social requirements (norms) of the classroom compatible with the work contexts and the social relationships (norms) of Native Hawaiian culture, based on the knowledge acquired though the prefatory naturalistic studies of Hawaiian child socialization. These compatibilities included creating a compatible small-group classroom organization, the use by teachers of compatible social reinforcement and social control techniques, and the promotion of compatible interactional patterns between teachers and children in the learning tasks. Furthermore, the program developers systematically altered KEEP in light of data obtained from continuous of monitoring the performance of the children. The third and nal stage exported KEEP into the public schools that serve Native Hawaiian children. KEEP worked closely with program evaluators to identify solutions for many of the unique educational problems faced by kindergarten through third-grade Hawaiian-American children, and to discover methods for disseminating these solutions to other Hawaiian schools. Even though the evaluation involved a multidisciplinary approach including theoretical perspectives from the elds of psychology, anthropology, education, and linguistics, it took 7 years before signicant improvement occurred (see Tharp et al., 2007).

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

363

The relevant point here is that the ABC principles outlined in the present manuscript not only provide an account of why this success could occur, it provides guidance regarding additional changes one might implement to further improve KEEP. We illustrate the practical use of the present account and predictions derived from it by working through the six examples below. 6. Summary By the present view, instrumental verbiage (e.g., speaking, reading, writing, and the like) constitutes a class of behaviors inuenced by its context and its consequences. The same is true of instrumental actions. The source of these consequences serves as a rough-and-ready way to differentiate declarative instrumental verbiage and procedural (or instrumental) action. Both declarative instrumental verbiage and instrumental actions are partially under the inuence of physical/abiotic contexts and consequences (food, water, danger, etc.) and partially under the inuence of social contexts and consequences (approval, disapproval, Altruistic Punishment, etc.). Although some physical/abiotic contexts and consequences may be involved, social contexts and consequences appear to be a primary inuence on adherence to social norms. A second important fact to derive from a functional perspective is the existence of multiple baselines. This label refers to the fact that several classes of behavior may unfold simultaneously. In Arizona, for example, one often sees individuals driving while talking on a cellular telephone. These drivers perform two distinguishable tasks: maintaining control of an automobile according to the rules of the road (instrumental action) and maintaining a conversation according to the rules of social interactions (instrumental verbiage). Some call this multitasking. A third important fact is that separable classes of behavior may be under the inuence of distinct consequences. To return to Arizona drivers, the consequences of violating the rules of the road (e.g., crashes, tickets) inuence instrumental actions associated with driving whereas consequences of violating social conventions (e.g., insulting a person, losing a business deal) inuence instrumental verbiage (e.g., voice modulation, polite responses) simultaneously. That is, distinguishable schedules of reinforcement operate simultaneously: one set of consequences inuence the behavior classied as driving (instrumental or procedural behavior) and another set of consequences inuence the behavior classied as talking on the telephone (instrumental verbiage or declarative behavior). If, as we argue, this scenario operates on most of, if not all adult education programs, then we have a reasonable way to predict the conditions under which those programs will fail. Moreover, we have several clues that might help us solve the ubiquitous and consistent problem of the failure of large-scale public education programs. 7. Applying these principles to psycho-educational programs 7.1. Two contingency sets Although we will not work out detailed examples, the principles outlined here permit us to describe, and provide examples of, conditions under which an education program will not perform up to expectations. Bluntly put, programs predicted to fall short of expectations provide support for declarative routines (instrumental verbiage) but do not provide support for appropriate procedural routines (instrumental action). These conditions establish context-specic social norms for particular ways of speaking about program goals. They produce appropriate instrumental verbiage ideas or principles because articulating them avoids social disapproval

and other aversive consequences as long as the person remains enrolled in the program; they do not, however, have procedures to stabilize newly established social norms or to stabilize appropriate instrumental action in the open environment. That is, although the Antecedents and the target Behaviors (the norms) articulated in the program may be clear, there are few, if any, consequences for failures to adhere to widely accepted Descriptive norms that is, to emit appropriate target Behaviors in the open environment. Hence, the theory predicts that programs offering no real consequences for failures to adhere to Descriptive norms for doing anything other than articulating appropriate ideas within the context of the program itself will not perform as expected. 7.2. Preventing initiation of adolescent smoking Extant meta-analyses of the outcome of school-based interventions designed to prevent the onset of adolescent cigarette smoking often use the causal model anchoring the program (program theory) as a classication tool, yielding, roughly, four approaches to program development (Wiehe, Garrison, Christakis, Ebel, & Rivara, 2005; Thomas & Perera, 2002; Rundall & Bruvold, 1988; Bruvold, 1993). These programs cover the gamut from programs targeting instrumental verbiage alone through those targeting instrumental action. Rational programs, for example, provide fact-based information on smoking and its delayed detrimental health consequences. Psychological programs attempt interventions targeting increased self-esteem and general life skills, such as decision-making. Social Inuence programs teach youth to recognize social settings where drug use may appear desirable and then provide practice mastering such situations without engaging in drug use. Contextual programs attempt to inuence the environmental consequences in various social settings (school or community) of youth, e.g. by changing school policies around tobacco use in addition to curriculum-based lessons. No class of program directly targets governance (norm adherence) in the open environment. These intervention models differ in respect to their major didactic methods, e.g. non-interactive methods such as lectures through teachers in information-based programs or interactive role-playing to acquire new social skills in social-inuence models (Tobler, 1986). Findings on the effectiveness of these interventions on self-described behavior generally report small to medium effect sizes (Tobler, 1986; Rooney & Murray, 1996; Hwang, Yeagley, & Petosa, 2004; Rundall & Bruvold, 1988), and small effect sizes for studies that included long-term follow-ups (Skara & Sussman, 2003). All available meta-analyses, however, conclude that Social Inuence and Contextual programs inuence relevant behavioral markers (mostly self-reported smoking, sometimes biologically validated) more than Rational or Psychological programs. One study (Tengs, Osgood, & Chen, 2001) used simulation models and sensitivity analyses to assess cost-effectiveness of providing school-based prevention curricula on a national scale and found favorable results indicating a positive return-of-investment under varying model assumptions, including a dissipation of effects after 14 years. Rational and psychological programs assume that increased factual knowledge of consequences of health-comprising behaviors is a necessary and sufcient condition for norm adherence in the open environment (i.e. generalize to other contexts outside the classroom). The analysis offered here asserts the assumption is false. Programs that teach factual knowledge, by denition, target declarative behaviors (instrumental verbiage) leaving both instrumental action and governance itself without support. Hence, the view predicts what is observed changes in declarative behaviors but little to none in instrumental action or governance. Similarly, Social Inuence and Contextual interventions provide direct

364

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

support for changes in instrumental verbiage, but also offer some, although indirect support for open environment changes in instrumental action and governance. Again, the view predicts what is observed direct changes in performance on assessments tapping declarative behavior and some change in those tapping instrumental action and governance. Although many recognize these assertions as fact (see e.g., Faggiano et al., 2008; Medley, Kennedy, OReilly, & Sweat, 2009; for reviews of various programs), and some report success of skillbased programs (e.g., Halford, Markman & Stanley, 2008), few describe principled theory-based reasons explaining why some programs are effective and others are apparently ineffectual. The commonality of ndings in several meta-analytical studies that interventions aimed at changing the way people talk about harmfulness of smoking succeeded in doing so (e.g. Hwang et al., 2004) and, at the same time, do not change instrumental action (and thereby governance) conrms the superiority of interactive, social-inuence-based interventions over psycho-educational ones. This empirical evidence, derived from decades of research on smoking prevention programs, is inconsistent with the assumption that increased factual knowledge of consequences of healthcomprising behaviors is a necessary and sufcient condition for norm adherence in the open environment. Moreover, the empirical evidence provides limited support for an ABC framework of norm adherence. 7.3. Promoting pro-environmental behaviors A vast literature indicates that human behavior contributes directly to environmental deterioration (e.g., Ponting, 1992). If this relation is to change, then human behavior must change. The majority of programs dedicated to this end assume a direct correspondence between information/education knowledge and behavior itself. That is, those creating programs in this eld assume governance (a strong correspondence between expressed norms (instrumental verbiage/declarative behavior) and instrumental action) emerges directly from appropriately trained instrumental verbiage. Problematically, current efforts to increase knowledge information-based programs that have been ongoing for the past 20 years suggest the assumption is false. These programs do not detectably increase human pro-environmental behavior (Hungerford & Volk, 1990; Frick, Kaiser, & Wilson, 2004; Kollmuss & Agyeman, 2002; Pooley & OConnor, 2000; Sia, Hungerford, & Tomera, 1985/86). According to the view presented here, programs that ignore instrumental action and governance while targeting various classes of instrumental verbiage (cast for example as awareness, attitude, and declarative knowledge) will not perform as expected. Moreover, the view predicts that programs that do not specify proper normative rules or provide ill-dened/inadequate consequences of governance by normative rules (i.e. if statements with no enforceable then statements) will come up short. Finally, the view suggests that psycho-educational interventions that do not adequately specify or provide a three-term contingencies supporting change in instrumental action (reinforcement for target Behavior) and changes in governance, contingencies that connect normative rules (the educational material/knowledge) with consequences for adherence or non-compliance (governance), will not produce promised behavioral changes in the open environment. In short, the view predicts that even though verbal statements may appropriately highlight connections among ecosystems, species, habitats, human impacts, large-scale consequences of human disturbances, and may even specify solutions that could remedy these problems, without enforceable behavioral consequences for norm adherence (following or breaking normative

rules), little change in instrumental action or governance will occur. Consider the fact that traditional environmental information/ education programs targeting measures of attitude (instrumental verbiage) have produced few demonstrable effects on instrumental behavior. The outcome of a study described by DiEnno and Hilton (2005) provides a clear example. These authors compared the outcome of a traditional psycho-educational program targeting expressed norms against the outcome a constructivist program targeting adherence to expressed norms that t within an individuals existing worldview and/or knowledge base. That is, the traditional program targeted and assessed declarative routines only; the constructivist program targeted both declarative and procedural routines. Given the foregoing theoretical work, it is no surprise that the traditional program produced no detectable effects on expressed social norms measured as knowledge or attitudes whereas the constructivist program produced signicant effects on both instrumental verbiage (expressed descriptive norms) and instrumental action (target Behavior). Consider the outcome of a comparable program offered by Dimopoulos, Paraskevopoulos, and Pantis (2008). These authors tested an environmental educational program that contained informational (declarative) modules along with activities requiring active participation, group cooperation, and project-based learning (procedural modules). The authors used a quasi-experimental, preand post-test design to assess changes in declarative and procedural routines. Furthermore, the authors included an active-learning process, which combined both traditional and constructivist processes described above. Results showed that only a subclass of the declarative routines changed (Dimopoulos et al., 2008). That is, some classes of instrumental verbiage (declarative routines reecting knowledge) increased whereas others (attitudes and verbal commitments to act in a pro-environmental way) did not (see also, Armstrong & Impara, 1991; Aivazidis, Lazaridou, & Hellden, 2006; Smith-Sebasto & Cavern, 2006). The authors did not take measures of either instrumental action or governance itself. It therefore remains uncertain if the program affected either instrumental action or governance. Clearly, environmental information/education programs have effectively changed the way that people talk about the environment. These programs have raised awareness of global warming, the necessity to recycle, and the prevalent need for conservation (e.g., Kaiser & Fuhrer, 2003; Kaiser, Ranney, Hartig, & Bowler, 1999). That is, these programs have changed the way that people talk about global warming, recycling, and conservation. Unfortunately, this awareness has not translated to pro-environmental instrumental actions. 7.4. Campaigns against sexual violence Despite mandatory knowledge-based anti-sexual violence education, one-in-three men and two-in-three women enrolled in college courses reported participating in a sexual crime, as either a victim or perpetrator, during the past year (Sisco & Figueredo, 2008). Knowledge-based psycho-educational campaigns designed to decrease not only reports of, but also the rate of sexual crime, encourage healthy sexual attitudes in an effort to create healthy sexual behavior. By the underlying model, expressed sexual attitudes causally inuence enacted sexual behavior. Hence, changing inappropriate attitudes will change action. As in other elds, most researchers assess changes in targeted attitudes by means of self-report. The attitudes most often targeted by these programs are: (1) reducing the acceptance of sexual aggression (2) discouraging the blaming of victims and (3)

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

365

promoting open discussion of sexual boundaries and contraception. Because these attitudes are widely known to be more socially desirable than others, it is not clear if the attitude change reported by these programs reect changes in personal values or is merely an artifact of training people to articulate socially desirable words. Perhaps more importantly, it is not clear open environment instrumental action or governance reects attitude change. Though anti-sexual violence psycho-educational programs consistently change instrumental verbiage interpreted as increased factual knowledge they do not reduce self-reported rape-supportive attitudes, questionable sexual intentions, or victim blaming (Breitenbecher & Scarce, 2001). Further, even when anti-sexual violence campaigns alter core sexual attitudes, the efforts appear futile self-reported beliefs do not correlate highly with self-reported sexually aggressive behavior (Gidycz et al., 2001; Kalof, 2000; Sisco, Becker, Figueredo, 2006). That is, open environment instrumental actions do not change detectably. 7.4.1. Individual differences in suitability for treatment One reason for the consistent failure of such programs is that individuals may obstruct the effectiveness of the campaign. In traditional language, for example, a psychopath is not likely to attend to a message appealing to ones conscience. Similarly, when Breitenbecher and Scarce (2001) inquired about the reasons awareness campaigns fail, students responded that, although the information was powerful, it was not applicable to them as individuals. Perhaps this is the result of a one size ts all approach. If only a small portion of the rape awareness information is personally relevant to each student in the audience, individual students would be more likely to disregard the relevance of the entire message than to reconsider their general feelings of invincibility (Festinger, 1957). By this view, unique phylogenetic, ontogenetic, and perceptual experiences compose each individuals extant antecedents and effective consequences (see glossary). Although normal social consequences may inuence the probability of a psychopaths instrumental verbiage temporarily, they are unlikely to inuence the probability of that same psychopaths instrumental actions in the open environment. A similar condition may exist for women and men. For example, women and men generally speak of sexuality in very different ways; a meta-analysis of 177 studies demonstrated signicantly more women than men report fear, guilt, or shame about sexuality and report feeling less comfortable with their sexual choices (Oliver & Hyde, 1993). Hence, the social consequences of adherence or non-compliance to injunctive sexual norms may differ (dramatically) for women and men. If this is the case, then programs designed specically for female or male sensitivity to governance failures in this realm may be an important component of any program aimed at decreasing sexual violence. 7.4.2. Norm adherence not linked to real world consequences The social norm that awareness training theoretically embodies (Do not commit sexual violence) differs from the normative rule perceived by the audience (Do not publicly endorse sexual violence). Currently, over 75% of students report being involved in sexual violence and yet only 6% of sexual crimes are reported to campus authorities (Sisco et al., 2006). Even fewer of reported crimes produce convictions Thus, non-adherence to the social norms, and even the formal legal rules, governing the inappropriateness of sexually coercive behaviors may not produce immediate aversive consequences with sufcient reliability to deter the behavior. Further, one could conjecture that positive attributes of sexual encounters stabilize the behavior. Moreover, any public endorsement of rape-supportive beliefs invokes immediate punitive responses (ranging social disapproval through legal retribu-

tion) from almost all public forums. Due to decades of awareness campaigns, rape-supportive public speech has become taboo; consequently, there is now so little variance in rape-related selfreported beliefs that self-report does not predict actual aggression (Loo & Thorpe, 1998). Hence, although verbal statements about this topic are subject to immediate social consequences, the real problem, the target behavior, is not. Thus, unique ABC relations surround targeted verbal behavior and the target Behavior and they appear to occur in parallel with one set of antecedents and consequences supporting the instrumental verbiage and another set supporting the instrumental action.12 Theoretically, what is needed is a consistent targeting of instrumental action (procedural) and instrumental verbiage (declarative routines). 8. Reconstructing and articulating principles of rule governance relative to successful social programs 8.1. Interventions with teeth Thus far, we have discussed information-only programs that have failed. We now turn to examples of programs that not only work, but illustrate (at least indirectly) the principles we have outlined herein. 8.2. Operation Ceasere Consider Operation Ceasere (e.g., Braga, Kennedy, Waring, & Piehl, 2001). The specied outcome of the program was to control and deter serious gang violence particularly youth homicides. To do so, the project designers posited . . . that crimes can be prevented when the costs of committing the crime are perceived by the offender to outweigh the benets of committing the crime (Gibbs, 1975; Zimring & Hawkins, 1973) (Braga et al., 2001, p. 201). To do so, the intervention occurred in two phases. During the rst, the working group implemented a set of activities to diminish the ow of rearms into the target area.13 During the second, the project designers held formal meetings and individual contact with gang members delivering a simple rule: (A) if in the city of Boston, (B) and if violent behavior occurred (that is, if a gang member hurt another person), (C) then, the authorities will pull every legal lever available. The pulling every lever consequences consisted of, for example, The authorities could disrupt street drug activity, focus police attention on low-level street crimes such as trespassing and public drinking, serve outstanding warrants, cultivate condential informants for medium- and long-term investigations of gang activities, deliver strict probation and parole enforcement, seize drug proceeds and other assets, ensure stiffer plea bargains and sterner prosecutorial attention, request stronger bail terms (and enforce them), and focus potentially severe federal investigative and prosecutorial attention on, for example, gang-related drug activity. Hence, the program designers set up an incentive structure through their rule and left it to the members of each gang to form and enforce gang-specic norms to prevent contact with that incentive structure. The program designers therefore created a social norm to be enforced internally within the gangs themselves, by applying
12 Brechner (1977) introduced the phrase superimposed schedules to describe one form of compound schedules a situation in which a single response leads to multiple consequences. Although it is attractive to think of rule governance in this way, the condition does not obtain. Instead, it appears that two distinct schedules are in effect at the same time each operating on distinct classes of instrumental behavior the target behavior and governance each class of behavior leading to its own unique set of consequences a situation that one might describe with the phrase simultaneous schedules. 13 Note that the working group did not expect to decrease the number of rearms already in gang hands, but instead to limit the supply of new rearms.

366

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369

external consequences collectively upon the gangs as selfregulating social units. Although the design of the intervention prevented strong causal inferences, as the authors express it, The [data pattern] shows a 63 percent reduction in the mean monthly number of youth homicide victims from a pretest mean of 3.5 youth homicides per month to a posttest mean of 1.3 youth homicides per month. This simple analysis suggests that Operation Ceasere was associated with a large reduction in youth homicides in Boston (see also Piehl, Kennedy, & Braga, 2000) (p. 204). 8.3. Smoking cessation As for the issue of youth smoking, several econometric studies converge on the nding that raising the price for cigarettes through tax increases results in stronger declines of participating in smoking among youth than adults: young smokers have a higher price elasticity of demand for cigarettes than older smokers. This effect may have two prongs: abruptly higher prices for cigarettes through tax increases as opposed to small-increment price raises by the tobacco industry, (Liang & Chaloupka, 2002) cause younger people who have already started to reduce their consumption (Chaloupka & Grossman, 1996) and deters those on the brink of becoming regular smokers to do so (Ross & Chaloupka, 2003). Lewit, Coate, and Grossman (1981) estimate that youth between 12 and 17 have a price elasticity of demand of stunning 1.2, or about 3 times more than adults on smoking participation. 8.4. Pro-environmental behavior An effective instrumental behavioral intervention was performed in Memphis, TN in the early 1990s. The project involved, . . . renement of penalty strategies to manage behavior, (Potter, Dwyer, & Lemming, 1995, p. 197). Prior to a complete reorganization of the court system, there was not a shared norm identifying pollution and other environmentally bad behaviors as important to reduce. Using functional analytic methodology, norms and contingencies were changed through an integrative, supportive, coherent infrastructure in the following ways: (1) a single court was instituted (with one judge) to hear environmental code infractions (resulting in expedient justice (without delay) and setting precedent with consistent, predictable consequences), (2) a reduction of inter-agency personnel involved with environmentalcode infractions thereby creating continuity of individuals involved with cases (resulting in shared social norms and positive reinforcement structures in those personnel), (3) signicant increase in penalties (nes for infractions) resulting in a signicant instrumental behavioral consequence (previous penalties were nominal and thus trivial), and (4) the judge presiding over the case follows through with personal site visits where previous infractions took place (thereby changing Antecedent conditions for all stakeholders and potential infractors). As a result, inspectors report a major increase in violators willingness to comply with environmental codes (Potter et al., 1995, p. 208). Inspectors report convergence among all enforcement personnel (shared norms), but also individuals expected to comply with regulations are now intimidated (again shared norms). 9. General discussion We reviewed the relevant literature on three distinct kinds of information-only or psycho-educational interventions in the elds of smoking prevention, pro-environmental behavior, and rape prevention, and critiqued the reconstructed program theory underlying each of these information-only interventions from the perspective of behavioral analytic theory. In each case, we

found there is insufcient attention to important distinctions between declarative and procedural routines and the contingent consequences needed to shape them. Most of these psychoeducational interventions rely on verbal consequences of instrumental verbiage (declarative routines), and do not provide social consequences for overt instrumental actions (procedural routines) which are presumably the ultimate targets of the program. Although, in many instances, providing social consequences for overt instrumental actions is far beyond the sphere of inuence of program designers, the pragmatics of this proposal are something to be worked out in the future. All we are indicating in the present argument is the need for this kind of real-world contingency for the program to work. We also reviewed relevant literature in a way that permits a bridge across this gap, namely by the principles of norm adherence (governance), which brings instrumental action under the inuence of verbally specied, clearly articulated injunctive normative rules. This principle indicates that it might be possible to establish contingencies in the open environment supporting reliable adherence to learn verbal normative rules. We have found, however, that none of the three areas of psycho-educational interventions reviewed implements the necessary procedures to accomplish this goal. To increase the probability of success, as dened by those implementing social programs, enhanced success may come hand-in-hand with enhanced program designs. Finally, we have shown by these specic illustrations how society can derive cumulative knowledge by attending to the cumulative history of social programs and their associated evaluation outcomes rather than proceeding program-by-program. We have used functional analytic theory as our metatheoretical framework to guide a critique of the reconstructed program theory underlying psycho-educational interventions. We are not claiming, however, this is the only perspective on this matter. Nor are we claiming this is a complete account of the theory needed to guide effective social programs. Instead, we merely offer it as an example of the kind of integration (consilience) that we can and must do if we are to create a truly cumulative body of program evaluation theory. If such a body of cumulative knowledge can be amassed, then it opens up the possibility that in the future these principles could be applied proactively to advise against programs, interventions, or treatment components that are doomed to failure, and to recommend others that are more likely to succeed. Acknowledgements This research was partially supported by personal funds from W. Jake Jacobs and A.J. Figueredo. The order of 2nd, 3rd, and 4th authorship was determined by lot. We thank Sacha D. Brown, Rafael Garcia, Lee Sechrest, Mei-Kuang Chen, Javier Espeleta and Adriana Quiros for comments, corrections, and careful thought regarding these ideas and their presentation. We, of course, take full responsibility for the nal product that you now hold in your hands. References
Aivazidis, C., Lazaridou, M., & Hellden, G. F. (2006). A comparison between a traditional and an online environmental education program. Journal of Environmental Education, 37(4), 4554. Armstrong, J. B., & Impara, J. C. (1991). The impact of an environmental education program on knowledge and attitude. Journal of Environmental Education, 22(4), 36 40. Barush, D. E., Kanter, J. W., Busch, A. M., Richardson, J. V., & Barnes-Holmes, D. (2007). The differential effects of instructions on dysphoric and nondysphoric persons. The Psychological Record, 57, 543554. Baum, W. H. (2004). Understanding behaviorism: behavior, culture, and evolution (2nd Edition). Wiley-Blackwell. Behavior: The environmental court as a contingency manager. Environment and Behavior, 27(2), 196212.

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369 Benjamini, Y., Lipkind, D., Horev, G., Fonio, E., Kafka, N., & Golan, I. (2010). Ten ways to improve the quality of descriptions of whole-animal movement. Neuroscience & Biobehavioral Reviews, 34, 13511365. Bloom, S. G. (2005). Lesson of a lifetime. Smithsonian Magazine. September, available from: http://www.smithsonianmag.com/history-archaeology/lesson_lifetime.html. Boring, E. G. (1950). A history of experimental psychology (2nd ed.). New York: AppletonCentury-Crofts. Braga, A. A., Kennedy, D. M., Waring, E. J., & Piehl, A. M. (2001). Problem-oriented policing, deterrence, and youth violence: an evaluation of bostons operation ceasere. Journal of Research in Crime and Delinquency, 38, 195225. Brechner, K. C. (1977). An experimental analysis of social traps. Journal of Experimental Social Psychology, 13, 552564. Breitenbecher, K. H., & Scarce, M. (2001). An evaluation of the effectiveness of a sexual assault education program focusing on psychological barriers to resistance. Journal of Interpersonal Violence, 16(5), 387407. Brownson, R. C., Gurney, J. G., & Land, G. H. (1999). Evidence-based decision making in public health. Journal of Public Health Management and Practice, 5, 8697. Bruvold, W. H. (1993). A metaanalysis of adolescent smoking prevention programs. American Journal of Public Health, 83, 872880. Campbell, D. T. (1971/1991). Methods for the experimenting society. American Journal of Evaluation, 12, 223260. Carpenter, J., Bowles, S., Gintis, H., & Hwange, S. (2009). Strong reciprocity and team production: theory and evidence. Journal of Economic Behavior & Organization, 71, 221232. Catania, A. C., Shimoff, E., & Matthews, B. A. (1989). An experimental analysis of rulegoverned behavior. In S. C. Hayes (Ed.), Rule-governed behavior: Cognition, contingencies, and instructional control (pp. 119150). New York, NY: Plenum. Cerutti, D. T. (1989). Discrimination theory of rule-governed behavior. Journal of the Experimental Analysis of Behavior, 51, 259276. Chaloupka, F., & Grossman, M. (1996). Price, tobacco control policies and youth smoking. Cambridge, MA: National Bureau of Economic Research. (Working Paper 5740). Cialdini, R. B., Reno, R. R., & Kallgren, C. A. (1990). A focus theory of normative conduct: recycling the concept of norms to reduce littering in public places. Journal of Personality and Social Psychology, 58, 10151026. Clayton, R. R., Cattarello, A. M., & Johnstone, B. M. (1996). The effectiveness of drug abuse resistance education (Project DARE): 5-year follow-up results. Preventive Medicine, 25(3), 307318. Cochran, G., & Harpending, H. (2009). The 10000 year explosion: how civilization accelerated human evolution. New York: Perseus Books. Cosmides, L., & Tooby, J. (1992). Cognitive adaptations for social exchange. In J. Barkow, L. Cosmides, & J. Tooby (Eds.), The adapted mind: Evolutionary psychology and the generation of culture. New York: Oxford University Press. de Quervain, D. G. F., Fischbacher, U., Treyer, V., Schellhammer, M., Schnyder, U., Buck, A., et al. (2004). The neural basis of altruistic punishment. Science, 305, 12541258. Delgado, M. R., Rita, L., Jou, R. L., LeDoux, J. E., & Phelps, E. A. (2009). Avoiding negative outcomes: tracking the mechanisms of avoidance learning in humans during fear conditioning. Frontiers in Behavioral Neuroscience, 3, 19. DeMaster, B., Reid, J., & Twentyman, C. (1977). The effects of different amounts of feedback on observers reliability. Behavior Therapy, 8, 317329. Deutsch, M., & Gerard, H. B. (1955). A study of normative and informational social inuence upon individual judgment. Journal of Abnormal and Social Psychology, 51, 629636. DiEnno, C. M., & Hilton, S. C. (2005). High school students knowledge, attitudes, and levels of enjoyment of an environmental education unit on nonnative plants. Journal of Environmental Education, 37(1), 1325. Dimopoulos, D., Paraskevopoulos, S., & Pantis, J. D. (2008). The cognitive and attitudinal effects of a conservation educational module on elementary school students. Journal of Environmental Education, 39(3), 4761. Doll, B., Jacobs, W. J., Sanfey, A., & Frank, M. (2009). Instructional control of reinforcement learning: a behavioral and computational investigation. Brain Research, 1299, 7494. Dreber, A., Rand, D. G., Fudenberg, D., & Nowak, M. A. (2008). Winners dont punish. Nature, 452, 348351. Dukes, R. L., Ullman, J. B., & Stein, J. A. (1996). Three-year follow-up of drug abuse resistance education (DARE). Evaluation Review, 20(1), 4966. Egas, M., & Riedl, A. (2008). The economics of altruistic punishment and the maintenance of cooperation. Proceedings of the Royal Society B, 275, 871878. Eichenbaum, H., & Cohen, N. J. (2004). From conditioning to conscious recollection: memory systems of the brain. New York: Oxford University Press. Eisenberger, N. I., & Lieberman, M. D. (2004). Why rejection hurts: a common neural alarm system for physical and social pain. Trends in Cognitive Sciences, 8, 294300. Eisenberger, N. I., Lieberman, M. D., & Williams, K. D. (2003). Does rejection hurt? An fMRI study of social exclusion. Science, 302, 290292. Ennett, S. T., Tobler, M. S., Ringwalt, C. T., & Flewelling, R. L. (1994). How effective is drug abuse resistance education? A meta-analysis of Project DARE outcomes evaluations. American Journal of Public Health, 84(9), 13941401. Eshkol, N., & Wachman, A. (1958). Movement notation. London: Weidenfeld and Nicolson. Faggiano, F., Vigna-Taglianti, F., Versino, E., Zambon, A., Borraccino, A., & Lemma, P. (2008). School-based prevention for illicit drugs use: a systematic review. Preventive Medicine, 46, 385396. Fehr, E., & Gaechter, S. (2002). Altruistic punishment in humans. Nature, 415, 137140. Festinger, L. (1957). A theory of cognitive dissonance. Stanford, CA: Stanford University Press. Gibbs, J. P. (1975). Crime, Punishment and Deterrence. New York: Elsevier.

367

Figueredo, A. J., Brumbach, B. H., Jones, D. N., Sefcek, J. A., Vasquez, G., & Jacobs, W. J. (2007). Ecological constraints on mating tactics. In G. Geher & G. F. Miller (Eds.), Mating intelligence: Sex, relationships and the minds reproductive system (pp. 335 361). Mahwah, NJ: Lawrence Erlbaum. Figueredo, A. J., Hammond, K. R., & McKiernan, E. C. (2006). A Brunswikian evolutionary developmental theory of preparedness and plasticity. Intelligence, 34(2), 211227. Figueredo, A. J., Wolf, P. S. A., Olderbak, S. G., Schlomer, G. L., & Garcia, R. A. Program evaluation. In T. D. Little (Ed.), The oxford handbook of quantitative methods. New York, NY: Oxford University Press, in press. Fishbein, M., Hall-Jamieson, K., Zimmer, E., von Haeften, I., & Nabi, R. (2002). Avoiding the boomerang: testing the relative effectiveness of antidrug public service announcements before a national campaign. American Journal of Public Health, 92, 238245. Fishbein, M., & Ajzen, I. (2010). Predicting and changing behavior: the reasoned action approach. New York: Psychology Press. Frick, J., Kaiser, F. G., & Wilson, M. (2004). Environmental knowledge and conservation behavior: exploring prevalence and structure in a representative sample. Personality and Individual Differences, 37, 15971613. Galizio, M. (1979). Contingency-shaped and rule-governed behavior: instructional control of human loss avoidance. Journal of the Experimental Analysis of Behavior, 31, 5370. Gantt, W. H. (1953). Principles of nervous breakdown schizokinesis and autokinesis. Annals of the New York Academy of Science, 56, 143163. General Accountability Ofce. (2003). Youth illicit drug use prevention (Report No. GAO-03-172R). Marjorie K.E.: Author (http://www.gao.gov/new.items/ d03172r.pdf). Gidycz, C. A., Layman, M. J., Rich, C. L., Crothers, M., Gylys, J., Matorin, A., et al. (2001). An evaluation of an acquaintance rape prevention program: impact on attitudes, sexual aggression, and sexual victimization. Journal of Interpersonal Violence, 16(11), 11201138. Gigerenzer, G., Hertwig, R., & Pachur, T. (Eds.). (2011). Heuristics: The foundations of adaptive behavior. New York: Oxford University Press. Golani, I. (1976). Homeostatic motor processes in mammalian interactions: a choreography of display. In P. G. Bateson & P. H. Klopfer (Eds.), Perspectives in ethology (Vol. 2). New York: Plenum. Halford, W. K., Markman, H. J., & Stanley, S. M. (2008). Strengthening couples relationships with education: Social policy and public health perspectives. Journal of Family Psychology, 22, 497505. Hackenberg, T. D., & Joker, V. R. (1994). Instructional versus schedule control of humans choices in situations of diminishing returns. Journal of Experimental Analysis of Behavior, 62, 367383. Hayes, S. C. (1993). Rule governance: basic behavioral research and applied implications. Current Directions in Psychological Science, 2, 193197. Hayes, S. C., Brownstein, A. J., Zettle, R. D., Rosenfarb, I., & Korn, Z. (1986). Rule-governed behavior and sensitivity to changing consequences of responding. Journal of the Experimental Analysis of Behavior, 45, 237256. Herrmann, B., Thoni, C., & Gachter, S. (2008). Antisocial punishment across societies. Science, 319, 13621367. Hojo, R. (2002). Effects of instructional accuracy on a conditional discrimination task. Psychological Record, 52, 493506. Hungerford, H. R., & Volk, T. L. (1990). Changing learner behavior through environmental education. Journal of Environmental Education, 21(3), 821. Hwang, M. S., Yeagley, K. L., & Petosa, R. (2004). A meta-analysis of adolescent psychosocial smoking prevention programs published between 1978 and 1997 in the United States. Health Education & Behavior, 31, 702719. Immordino-Yang, M. H., McColla, A., Damasio, H., & Damasio, A. (2009). Neural correlates of admiration and compassion. Proceedings of the National Academy of Science, 106, 80218026. Jackson, P. L., Meltzoff, A. N., & Decety, J. (2005). How do we perceive the pain of others? A window into the neural processes involved in empathy. NeuroImage, 24, 771 779. Jacobs, W. J., Blackburn, J. R., Buttrick, M., Harpur, T. J., Kennedy, D., Mana, M. J., et al. (1988). Observations. Psychobiology, 16, 319. Joyce, J. H., & Chase, P. N. (1990). Effects of response variability on the sensitivity of rule-governed behavior. Journal of the Experimental Analysis of Behavior, 54, 251 262. Kantor, J. R. (1970). An analysis of the experimental analysis of behavior (TEAB). Journal of the Experimental Analysis of Behavior, 13, 101108. Kaiser, F. G., & Fuhrer, Ur s. (2003). Ecological behaviors dependency on different forms of knowledge. Applied Psychology: An International Review, 52(4), 598613. Kaiser, F. G., Ranney, M., Hartig, T., & Bowler, P. A. (1999). Ecological behavior, environmental attitude, and feelings of responsibility for the environment. European Psychologist, 4(2), 5974. Kalof, L. (2000). Vulnerability to sexual coercion among college women: a longitudinal study. Gender Issues, 18(Fall), 4758. Kamin, L., Brimer, C., & Black, A. (1963). Conditioned suppression as a monitor of fear of the CS in the course of avoidance training. Journal of Comparative and Physiological Psychology, 56(3), 497501. Kaufman, A., Baron, A., & Kopp, R. E. (1966). Some effects of instructions on human operant behavior. Psychonomic Monograph Supplements, 1, 243250. Kent, R. N., Kanowitz, J., OLeary, K. D., & Cheiken, M. (1977). Observer reliability as a function of circumstances of assessment. Journal of Applied Behavior Analysis, 10, 317324. Kent, R. N., OLeary, K. D., Diament, C., & Dietz, A. (1974). Expectation biases in observational evaluation of therapeutic change. Journal of Consulting and Clinical Psychology, 42, 774780.

368

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369 Sherif, M., Harvey, O. J., White, B. J., Hood, W. R., & Sherif, C. W. (1961). Intergroup conict and cooperation: the Robbers Cave experiment. Norman: University of Oklahoma Book Exchange. Shuller, D. Y., & McNamara, J. R. (1976). Expectancy factors in behavioral observation. Behavior Therapy, 7, 519527. Sia, A. P., Hungerford, H. R., & Tomera, A. N. (1985/86). Selected predictors of responsible environmental behavior: an analysis. Journal of Environmental Education, 17(2), 3140. Sisco, M. M., & Figueredo, A. J. (2008). Similarities between men and women in nontraditional aggressive sexuality: prevalence, novel approaches to assessment and treatment applications. Journal of Sexual Aggression, 14(3), 253266. Sisco, M. M., Becker, J. V., & Figueredo, A. J. (2006). Gender stereotypes, rape myths, and experiences of sexual victimization and perpetration among male and female college students. Available from authors, in submission to ATSA: Chicago, IL. Skara, S., & Sussman, S. (2003). A review of 25 long-term adolescent tobacco and other drug use prevention program evaluations. Preventive Medicine, 37, 451474. Skinner, B. F. (1938). The behavior of organisms: an experimental analysis. Oxford, England: Appleton-Century. Smeets, P. M., Dymond, S., & Barnes-Homes, D. (2000). Instructions, stimulus equivalence, and stimulus sorting: effects of sequential testing arrangements and a default option. The Psychological Record, 50, 339354. Smith-Sebasto, N. J., & Cavern, L. (2006). Effects of pre- and posttrip activities associated with a residential environmental education experience on students attitudes toward the environment. Journal of Environmental Education, 37(4), 317. Solomon, R., & Wynne, L. (1953). Traumatic avoidance learning: acquisition in normal dogs. Psychological Monographs, 67, 291302. Solomon, R., & Wynne, L. (1954). Traumatic avoidance learning: the principles of anxiety conservation and partial irreversibility. Psychological Review, 61, 353385. Solomon, R., Kamin, L., & Wynne, L. (1953). Traumatic avoidance learning: the outcomes of several extinction procedures with dogs. The Journal of Abnormal and Social Psychology, 48(2), 291302. Starr, M., & Mineka, S. (1977). Determinants of fear over the course of avoidance learning. Learning and Motivation, 8(3), 332350. Taplin, P. S., & Reid, J. B. (1973). Effects of instructional set and experimenter inuence on behavior reliability. Child Development, 44, 547554. Tees, R. C., & Werker, J. F. (1984). Perceptual exibility: maintenance or recovery of the ability to discriminate non-native speech sounds. Canadian Journal of Psychology, 38, 579590. Teitelbaum, O., Benton, T., Shah, P. K., Prince, A., Kelly, J. L., & Teitelbaum, P. (2004). Eshkol-Wachman movement notation in diagnosis: the early detection of Aspergers syndrome. Proceedings of the National Academy of Sciences of the United States of America, 101, 1190911914. Tengs, T. O., Osgood, N. D., & Chen, L. L. (2001). The cost-effectiveness of intensive national school-based anti-tobacco education: results from the tobacco policy model. Preventive Medicine, 33, 558570. Tharp, R., & Gallimore, R. (1979). The ecology of program research and development: a model of evaluation succession. In Sechrest, L. B., West, S. G., Phillips, M. A., Redner, R., & Yeaton, W. (Eds.), Evaluation studies review annual (Vol. 4, pp. 3960).Beverly Hills, CA: Sage Tharp, R., & Gallimore, R. (1982). Inquiry process in program development. Journal of Community Psychology, 10(2), 103118. Tharp, R. G., Jordan, C., Speidel, G. E., Au, K. H., Klein, T. W., Calkins, R. P., et al. (2007). lili: Multidisciplinary Education and native Hawaiian children: revisiting KEEP. Hu Research on Hawaiian Well-Being, 4(1), 269317. Thomas, R., & Perera, R. (2002). School-based programmes for preventing smoking. The Cochrane Database of Systematic Reviews, 4: 14651858. Tobler, N. S. (1986). Meta-analysis of 143 adolescent drug prevention programs: quantitative outcome results of program participants compared to a control or comparison group. Journal of Drug Issues, 16, 537567. Trivers, R. L. (1971). The evolution of reciprocal altruism. Quarterly Review of Biology, 46, 3557. Ullman, M. T. (2001). A neurocognitive perspective on language: the declarative procedural model. Nature Reviews Neuroscience, 2, 717726. UNESCO (1975). The Belgrade charter: a global framework for environmental education. Retrieved April 11, 2011 from http://www.envir.ee/orb.aw/class=le/ action=preview/id=1011467/The+Belgrade+Charter.pdf. UNESCO (1978). Final report intergovernmental conference on environmental education. Organized by UNESCO in Cooperation with UNEP, Tbilisi, USSR, 14-26 October 1977. Paris: UNESCO ED/MD/49. Weiss, C. H. (1997). Theory-based evaluation: past, present, and future. New Directions for Evaluation, 76, 4155. Weiss, C. H. (1998). Evaluation: methods for studying programs and policies (2nd ed.). Englewood Cliffs, N.J Prentice Hall. Werker, J. F., & Tees, R. C. (2005). Speech perception as a window for understanding plasticity and commitment in language systems of the brain. Developmental Psychobiology, 46, 233234. West-Eberhard, M. J. (2003). Developmental plasticity and evolution. Oxford University Press: New York. Wiehe, S. E., Garrison, M. M., Christakis, D. A., Ebel, B. E., & Rivara, F. P. (2005). A systematic review of school-based smoking prevention trials with long-term follow-up. Journal of Adolescent Health, 36, 162169. Wildman, B. G., Erickson, M. T., & Kent, R. N. (1975). The effect of two training procedures on observer agreement and variability of behavior ratings. Child Development, 46, 520524. Wood, W., & Neal, D. (2007). A new look at habits and the habit-goal interface. Psychological Review, 114, 843863.

Kim, H., Shimojo, S., & ODoherty, J. P. (2006). Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. Public Library of Science: Biology, 4, e233doi:10.1371/journal.pbio.0040233. Konorski, J. (1967). Integrative activity of the brain. Chicago: University of Chicago Press. Kollmuss, A., & Agyeman, J. (2002). Mind the gap: why do people act environmentally and what are the barriers to pro-environmental behavior? Environmental Education Research, 8(3), 239260. Krug, E. G., Mercy, J. A., Dahlberg, L. H., & Zwi, A. B. (2002). The world report on violence and health. The Lancet, 360, 10831088. Leeuw, F. L. (2003). Reconstructing program theories: methods available and problems to be solved. American Journal of Evaluation, 24, 520. LeFrancois, J. R., Chase, P. N., & Joyce, J. H. (1988). The effects of a variety of instructions on human xed-interval performance. Journal of the Experimental Analysis of Behavior, 49, 383393. Lewit, E. M., Coate, D., & Grossman, M. (1981). The effects of government regulations on teenage smoking. Journal of Law and Economics, 24, 545569. Liang, L., & Chaloupka, F. J. (2002 Feb). Differential effects of cigarette price on youth smoking intensity. Nicotine & Tobaco Research, 4(1), 109114 PubMed PMID: 11906687. Lippman, L. G., & Meyer, M. E. (1967). Fixed-interval performance as related to instructions and to subjects verbalizations of the contingency. Psychonomic Science, 8, 135136. Lipsey, M. W. (1993). Theory as method: small theories of treatments. New Directions for Program Evaluation, 57, 538. Loo, R., & Thorpe, K. (1998). Attitudes towards womens roles in society: a replication after 20 year. Sex Roles, 39(1112), 903912. Maier, N. R. F. (1949). Frustration, the study of behavior without a goal. New York: McGraw-Hill. Masserman, J. H. (1943). Behavior and neurosis. Chicago: University of Chicago Press. Medley, A., Kennedy, C., OReilly, K., & Sweat, M. (2009). Effectiveness of peer education interventions for HIV prevention in developing countries: a systematic review and meta-analysis. AIDS Education and Prevention, 21, 181206. Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371378. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167202. Mineka, S. (1979). The role of fear in theories of avoidance learning, ooding, and extinction. Psychological Bulletin, 86, 9851010. Moors, A., & De Houwer, J. (2006). Automaticity: a theoretical and conceptual analysis. Psychological Bulletin, 132, 297326. OHora, D., Barnes-Holmes, D., Roche, B., & Smeets, P. (2004). Derived relational networks and control by novel instructions: a possible model of generative verbal responding. The Psychological Record, 54, 437460. Ohtsuki, H., Iwasa, Y., & Nowak, M. A. (2009). Indirect reciprocity provides a narrow margin of efciency for costly punishment. Nature, 457, 7982. Oliver, M. B., & Hyde, J. S. (1993). Gender differences in sexuality: a meta-analysis. Psychological Bulletin, 117(1), 146154. Piehl, A. M., Kennedy, D. M., & Braga, A. A. (2000). Problem Solving and Youth Violence: An Evaluation of the Boston Gun Project. American Law and Economics Review, 2, 58106. Pellis, S. (2010). Conservative motor systems, behavioral modulation and neural plasticity. Behavioural Brain Research, 214, 2529. Pfaus, J. G., Blackburn, J. R., Harpur, T. J., MacDonald, M. A., Mana, M. J., & Jacobs, W. J. (1988). Has psychology ever been a science of behavior? A comment on Skinner. American Psychologist, 43, 821822. Ponting, C. (1992). A green history of the world: the environment and the collapse of great civilizations. New York: St Martins Press. Pooley, J. A., & OConnor, M. (2000). Environmental education and attitudes: emotions and beliefs are what is needed. Environment and Behavior, 32(5), 711723. Potter, L. E., Dwyer, W. O., & Lemming, F. (1995). Encouraging proenvironmental behavior: the environmental court as a contingency manager. Environment and Behavior, 27(2), 196212. Rand, D. G., Ohtsuki, H., & Nowak, M. A. (2009). Direct reciprocity with costly punishment: generous tit-for-tat prevails. Journal of Theoretical Biology, 256, 4557. Reid, J. B. (1970). Reliability assessment of observation data: a possible methodological problem. Child Development, 41, 11431150. Rogers, P. J. (2007). Theory-based evaluation: reections ten years on. New Directions for Evaluation, 114, 6367. Rogers, P. J., Petrosino, A., Huebner, T. A., & Hacsi, T. A. (2000). Program theory evaluation: practice, promise, and problems. New Directions for Evaluation, 87, 513. Romanczyk, K. G., Kent, R. N., Diament, C., & OLeary, K. D. (1973). Measuring the reliability of observational data: a reactive process. Journal of Applied Behavior Analysis, 6, 175184. Rooney, B. L., & Murray, D. M. (1996). A meta-analysis of smoking prevention programs after adjustment for errors in the unit of analysis. Health Education Quarterly, 23, 4864. Ross, H., & Chaloupka, F. J. (2003). The effect of cigarette prices on youth smoking. Health Economics, 12(3), 217230 PubMed PMID: 12605466. Rundall, T. G., & Bruvold, W. H. (1988). A meta-analysis of school-based smoking and alcohol-use prevention programs. Health Education Quarterly, 15, 317334. Satcher, M. D. (2001). Surgeon general of the United States. Youth violence: a report of the Surgeon General 2001. Prevention and Intervention, box 52. http://www.surgeongeneral.gov/library/youthviolence/chapter5/sec4.html. Schaffer, L. S. (1983). Toward Pepitones vision of a normative social psychology: what is a social norm? Journal of Mind and Behavior, 4, 275294.

W.J. Jacobs et al. / Evaluation and Program Planning 35 (2012) 354369 Zimbardo, P. G. (2007). The Lucifer effect: understanding how good people turn evil. New York, NY, US: Random House. Zimbardo, P. G. (2006). On rethinking the psychology of tyranny: the BBC prison study. British Journal of Social Psychology, 45, 4753. Zimring, F. E., & Hawkins, G. (1973). Deterrence: The Legal Threat in Crime Control. Chicago: University of Chicago Press.

369

Variable interval schedule reinforcement (VI): The reinforcer occurs after the organism emits the target Behavior and an unpredictable amount of time has elapsed. Concurrent schedules of reinforcement: The spatio-temporal relations between target Behaviors and contingent events that inuence the strength of those behaviors (probability, latency, pattern, etc.). These schedules of reinforcement are simultaneously available to the organism so the organism can emit target Behavior appropriate to either or both schedules.

Glossary
We provide this glossary because many of the words used here as theoretical constructs carry excessive surplus meaning. Rule: Instrumental verbiage that points to environmental contingencies. Takes an explicit or implicit if A, and if B, then C form. Rules set an occasion for instrumental action and Governance. May be called pointers. Social norms: A subclass of rules pointing to socially acceptable and unacceptable social if A, and if B, then C contingencies. Antecedent conditions: From the Latin What precedes or to go before. This includes the history of the species, the developmental history, extant anatomical and physiological states as they interplay with the extant environment. Instrumental: Serving as a tool. Instrumental behavior: What behavior as a tool accomplishes. Hence, the outcome of an instrumental behavior denes it. Often shortened to Behavior. Instrumental action: A subclass of instrumental Behavior classied by its form and measured by its outcome exclusively usually as environmental work. Often shortened to Operant. Verbiage: A subclass of instrumental behavior classied by its form (vocalization, writing, gesticulation, facial expression, etc.) and measured by its outcome exclusively usually as an inuence on conspecics. Although, according to the Oxford English Dictionary, the rst meaning of verbiage has negative connotations, we accept only the second entry as our technical denition: Diction, wording, verbal expression. We intend no other meaning. Governance/Adherence/Compliance/Obedience: A subclass of instrumental behavior measured by its t between environmental contingencies as specied by a rule and targeted instrumental behavior (t between ABC and rule specication). Often shortened to governance or adherence. Effective consequent: An event contingent on instrumental behavior, following it within 12 s and changing the subsequent strength (probability, latency, etc.) of that behavior. Often shortened to Consequent or Consequences. Schedule of reinforcement: The spatio-temporal relation between a target Behavior and contingent events that inuence the strength of the behavior (probability, latency, pattern, etc.). In this case, reinforcement refers to events that increase or decrease the strength of the behavior. Schedules of reinforcement come in two major forms: Continuous schedule of reinforcement (CRF): The reinforcer occurs immediately each time the organism emits the target Behavior. Also known as continuous reinforcement. Partial schedule of reinforcement: The reinforcer does not occur each time the organism emits the target Behavior. Partial schedules of reinforcement come in four major avors: Fixed ratio schedule of reinforcement (FR): The reinforcer occurs after the organism emits the target Behavior a xed number of times. Fixed interval schedule of reinforcement (FI): The reinforcer occurs after the organism emits the target Behavior and after a xed amount of time has elapsed. Variable ratio schedule of reinforcement (VR): The reinforcer occurs after the organism emits the target Behavior an unpredictable number of times. W. Jake Jacobs, Ph.D., is a Professor of Psychology and Psychiatry, Fellow of Sports Medicine, and Director of the Anxiety Research Group at the University of Arizona. He serves as a core member of the Ethology and Evolutionary Psychology and Cognitive and Neural Systems programs, and afliates with the Clinical Psychology program there. He serves as an Action Editor for Traumatology and has served on editorial boards of journals such as Rehabilitation Psychology and Psychological Review. Dr. Jacobs worries about proximate and ultimate theories of the etiology of contextualized anxiety disorders and stress, and relations between Hot/Cool Neural Systems and rule governance in regulating emotions. Melissa Sisco, Ph.D., is currently doing her clinical internship at the Chicago Psychiatric Center, University of Illinois where she is completing the nal requirements for a joint Ph.D. in Clinical Psychology and Psychology, Policy and Law. Ms. Sisco has worked in the area of sexual violence for the past 10 years specializing in institutional program development and evaluation of sexual violence prevention and treatment. She has worked with a variety of agencies in efforts to address the sequelae of sexual violence including international court, corrections, restorative justice, and mental health providers. Dawn Hill, Ph.D., teaches environmental and evolutionary psychology, biopsychology and personality at the University of Arizona. She is also a Conservation Psychology researcher and consultant. Dr. Hills research areas include individual and group conservation behavior, environmental education effectiveness and program evaluation, community conservation efforts, and the intersection of environmental science and law. Her primary emphases are in evolutionary and behavioral psychology, which brings a unique perspective in her eld, as well as investigating the strong effect of situational forces on conservation behavior and education. Frederic Malter, Ph.D., combines perspectives of social psychology and program evaluation to improve evaluation practice. He received his Ph.D. in program evaluation by examining outcomes of school-based anti-smoking education. Dr. Maters work focuses on the application of quantitative methods to support decision makers in the improvement of social programs and policies. He is interested in quantitative research on programs aiming to ameliorate social ailments. He is a member of the American. Evaluation Association (AEA), and the German Evaluation Society (DGEval). Dr. Malter has presented at numerous regional, national and international conferences and facilitated workshops around evaluation methods and practice. Aurelio Jose Figueredo, Ph.D., is a Professor of Psychology, Family Studies and Human Development at the University of Arizona. Dr. Figueredo serves as Director of the graduate program in Ethology and Evolutionary Psychology, an inter-disciplinary program integrating the studies of comparative psychology, ethology, sociobiology, and behavioral ecology, genetics, and development. He currently serves as a member of the Board of Directors of the Evaluation Group for Analysis of Data. His research interests lie in the evolutionary psychology and behavioral development of life history strategy, sex, violence, and the quantitative ethology and social development in human and nonhuman organisms.