Vous êtes sur la page 1sur 9

An Experimental Analysis of Steady-State Response Rate Components on

Variable Ratio and Variable Interval Schedules of Reinforcement


Phil Reed
Swansea University
Three experiments explored a novel approach to analyzing the different components of response rate that
are produced by exposure to free-operant schedules of reinforcement. It has been suggested that overall
response rate comprises a tendency to initiate responding, and to continue to respond once the bout is
initiated. Previous post hoc analyses of interresponse times (IRT) data have suggested several features of
these different aspects of responding that the current experimental procedure broadly confirmed.
Increasing the size of a variable interval (VI) schedule decreases the number of burst-initiation
responses, but has less effect on responding once the burst has been initiated (Experiment 1); that the
major difference between a variable ratio (VR) schedule and a VI schedule, is not in the number of
burst-initiation responses, but in the number of within-burst responses, with shorter interresponse
times that are emitted, with greater numbers of such within-burst responses being emitted on a VR
schedule (Experiments 2 and 3).
Keywords: response components, response initiation, burst responses, variable ration, variable interval,
rats
Response rates maintained by variable ratio (VR) schedules of
reinforcement are typically higher than those maintained by vari-
able interval (VI) schedules of reinforcement (Ferster & Skinner,
1957; Peele, Casey, & Silberberg, 1984; Zuriff, 1970). This find-
ing is one of some generality across species, having been noted in
rats (e.g., Dickinson, Peters, & Schechter, 1984; Reed, 2007),
pigeons (e.g., Peele et al., 1984; Ferster & Skinner, 1957), and
humans (e.g., Dack, McHugh, & Reed, 2009; Raia, Shillingford,
Miller, & Baier, 2000). Apart from the strength of the empirical
finding, this effect has been the source of a number of therapeutic
applications (see Lattal & Neef, 1996, for a review) and has
formed the basis of a number of investigations into the factors that
control free-operant responding (e.g., Baum, 1973, 1993; Morse,
1966; Reed, 2007).
Theoretical interpretations of this effect tend to focus either
on the molecular factors that control behavior, such as the
reinforcement of interresponse times (IRT; e.g., Morse, 1966),
or the molar features of the contingencies, such as the feedback
function relating the rate of response to the obtained rate of
reinforcement (e.g., Baum, 1973). In such experiments, the
overall rate of response is related to some feature of the con-
tingency, such as the length of the reinforced IRT, or the nature
of the feedback function, in order to determine the degree of
control which that aspect of the environment has over behavior.
Typically, such studies have shown that nonhuman rates of
responding on free-operant schedules of reinforcement are
strongly related to the molecular features of the environment
(e.g., Cole, 1999; Morse, 1966; Peele et al., 1984). However,
there also is some evidence, obtained from contingencies other
than VR and VI schedules, that the molar feedback function can
control rates of responding not only in humans (McDowell &
Wixted, 1986; Reed, 2007), but also in nonhumans (Bowers,
Hill, & Palya, 2008; Reed, 2007). That different aspects of the
environment can control behavior suggests that a strict molar
versus molecular dichotomy may be a false one, and that both
aspects of the environment will control behavior under different
conditions.
One feature of these theoretical debates is the assumption that
rate of response can be treated as a unitary dependent variable, and
that the fluctuation in the rate of response can be unambiguously
attributed to a single process, or to a set of processes. However,
there is an extensive literature that questions whether response rate
per se is a strong measure of the degree of learning about a single
aspect of the environment (Blough, 1963; Nevin, 1979; Pear &
Rector, 1979; Reed, Schachtman, & Hill, 1988). Some have sug-
gested that overall response rate may be the product of the devel-
opment of units of behavior, which consist of several integrated
responses (see Bacha-Mendez, Reid, & Mendoza-Soylovna, 2007;
Reed & Morgan, 2006). In the case of a VR schedule, such a unit
may comprise a group of responses, whereas, on a VI schedule this
unit may comprise a response pattern that is characterized by a
long IRT (e.g., see Morse, 1966). It may be that such units are
emitted with roughly equal probability on a VR and a VI schedule,
but as units on the former schedule contain more individual re-
sponses, the rate of response is higher on this schedule than on the
VI schedule (see Shull & Grimes, 2003). If this were the case, then
the overall rate of response may not be a completely ideal measure
This article was published Online First August 16, 2010.
Phil Reed, Department of Psychology, Swansea University.
These data were first presented at the First Conference of the European
Association for Behavior Analysis, Parma, Italy, 2003. Thanks are due to
Lisa A. Osborne for her support.
Correspondence concerning this article should be addressed to Phil
Reed, Department of Psychology, Swansea University, Singleton Park,
Swansea, SA2 8PP, United Kingdom. E-mail: p.reed@swansea.ac.uk
Journal of Experimental Psychology: 2010 American Psychological Association
Animal Behavior Processes
2011, Vol. 37, No. 1, 19
0097-7403/10/$12.00 DOI: 10.1037/a0019387
1
of learning on such schedules, although it should be acknowledged
that its use certainly has brought a number of advances in theo-
retical understanding.
More generally, such a view may be taken to suggest that overall
response rate may be a product of periods of responding that are
separated by periods of disengagement from responding (Blough,
1963; Pear & Rector, 1979; Shull, Gaynor, & Grimes, 2001). A
number of recent articles have attempted to analyze free-operant
responding in such a manner and to explore the consequences of
adopting such a view for the understanding of schedule perfor-
mance (e.g., Shull, 2004; Shull et al., 2001; 2002; Shull & Grimes,
2003). These studies have used an approach in which the respond-
ing of organisms on various schedules of reinforcement is ana-
lyzed in terms of their patterns of IRTs. The IRTs are displayed as
a log survivor plots; that is, the proportion of IRTs emitted (rela-
tive to the opportunity to emit that IRT) that are longer than any
given interval, are plotted against that interval. This method of
displaying the IRTs has the potential to produce a number of
different patterns of data. For example, if responding is randomly
emitted at a constant rate across a session, then the log survivor
plot would be a decreasing straight line with a simple exponential
decay (see Shull et al., 2001, p. 250, for discussion). However,
Shull et al. (2001, p. 250) noted that a feature of the data which
emerged when such plots are produced for rats nose-poke re-
sponses, was that the slope of the line was not uniform, but rather
comprised a sharply decreasing initial portion, followed by a
portion with a shallower negative gradient. This finding has been
taken to suggest that there are two different types of responding in
operation: a set of shorter IRTs, which are taken to reflect within-
burst responding; and a set of longer IRTs, which are taken to
reflect burst-initiation responses.
In a series of studies, it has been suggested that the two types of
responding (burst-initiation and within-burst responses) are
operated on by different aspects of the contingencies. For example,
the rate of burst-initiation responses has been noted to increase
with increasing rates of reinforcement, but within-burst re-
sponses are not so clearly affected by this aspect of the contin-
gency (Shull et al., 2001; Shull, 2004). That is, increasing the size
of a VI schedule (i.e., reducing the rate of reinforcement) decreases
the number of burst-initiation responses, but has less effect on
responding once the burst has been initiated. Moreover, it has been
suggested that the major difference between a VR schedule and a
VI schedule, is not in the number of burst-initiation responses,
but in the number of shorter within-burst responses that are
emitted (see Bowers et al., 2008; Shull et al., 2001; Shull &
Grimes, 2003); with greater numbers of within-burst responses
being emitted on a VR schedule. These results have lead to the
claim that burst-initiation responses are controlled by motiva-
tional factors (e.g., rate of reinforcement), and the within-burst
responses are controlled by schedule factors (e.g., reinforcement of
particular IRTs). This suggestion mirrors the distinction between
the strengthening and shaping properties of reinforcement made by
Morse (1966; see Shull et al., 2001).
Obviously, such a finding has many implications of the under-
standing of performance on free-operant schedules of reinforce-
ment, and is certainly worthy of further study. The form of anal-
yses suggested by Shull et al. (2001, 2002; see also Shull &
Grimes, 2003; Shull, 2004) is clearly ingenious, but there are a
number of problems that remain with these demonstrations. First,
a problem with the log survivor plot analyses is that they will not
provide a good estimate of the numbers of burst-initiation and
within-burst responses if the assumed exponential model does
not fit the data well, and/or if the distribution of burst-initiation
responses is similar to the distribution of within-burst responses.
Although these assumptions may fit data derived from rats nose-
poking, they may well not fit other response systems (see Kessel
& Lucke, 2008; Kulubekova & McDowell, 2008). Second, such
potential problems with the log survivor analyses, as outlined
above, have been documented, to some extent, in practice. For
example, Bowers et al. (2008) have suggested that identification of
such break points is not always easy for VI schedules, and,
moreover, that VR schedules may produce several such break
points. Furthermore, it has been suggested that, for some re-
sponses, such as the nose-poke of the rat, the break point may be
easily identifiable (Shull et al., 2001), but that for other responses,
such as the more commonly studied lever-pressing in rats, there
may be less clear points of inflection (see Shull & Grimes, 2003);
and that key-pecking in pigeons may be similarly resistant to a
straightforward analyses (see Bennett, Hughes, & Pitts, 2007;
Bowers et al., 2008).
Thus, a number of potentially important findings emerge from
the literature on the constituents of response rate; that is, burst-
initiation is dependent on rate of reinforcement, and effect that is
especially noticeable in VI schedules; increasing the VR schedule
value increases to number of within-burst responses; and VR and
VI schedules produce similar numbers of burst-initiation re-
sponses, but differ in the number of within-burst responses.
However, these suggestions may substantiated by a complimentary
experimental approach in order to corroborate and explore the
suggestions made on the basis of the analytic approach outlined
above. One potential approach to achieve this goal may be pro-
vided by the development of an experimental approach based on
that suggested by Mechner (1992), in terms of the revealed
operant. In this approach, two discrete manipulanda are provided
to the subject, a response to one manipulandum marks the start of
a response, which is then conducted on another manipulandum.
This clearly demarks the burst-initiation, from within-burst,
responses. A similar procedure has been conducted, on a small
scale, by Pear and Rector (1979) using pigeons as subjects. In this
study, pigeons were required to make one response (mounting a
small platform), which would gain access to an illuminated key,
which they could then peck to produce grain, according to a series
of interval schedules.
In the current series of experiments, rat subjects will be pre-
sented with two levers. At the start of a session, one of the levers
(the burst-initiation lever) will have a light, located above the
lever, illuminated. A response to this lever will be taken to mark
the start of a response, and will extinguish the light over that lever.
A light located above another, spatially distinct, lever will then be
illuminated, and that lever will be operative for the schedule to be
completed. If the rats cease responding for a period of time prior
to the schedule being completed, this is taken as the burst being
terminated (Mellgren & Elsmore, 1991). The light above the
response-burst lever will be extinguished, the lever will cease to be
operative, and the light above the burst-initiation lever will be
illuminated again. This way the numbers of burst-initiating, and
within-burst responses, can be clearly demarked from one an-
other. Of course, the length of time without a response that
2
REED
determines the end of a burst is an arbitrary (albeit preexperimen-
tally defined) criterion. A value of 5s was chosen for all of these
experiments based on previous data from rats lever-pressing that
suggested the point of inflection occurred around 5s for this type
of response (Bowers et al., 2008; Shull & Grimes, 2003).
Using this procedure it was hoped to established whether the
observations made from the post hoc analysis of IRT data by Shull
et al. (2001; see also Shull & Grimes, 2003), and noted above,
would be corroborated.
Experiment 1
In terms of responding on VI schedules of reinforcement, pre-
vious analyses of IRT patterns has suggested that within-burst
responding is not largely (or at least not clearly) altered by an
increase in the rate of reinforcement, but that the numbers of
burst-initiations are increased by this manipulation (Pear & Rector,
1979; Shull et al., 2001; although later work has noted a modest
positive relation between bout length and rate of reinforcement on
VI schedules; see Shull, Grimes, & Bennett, 2004). This is a
reasonably well established finding and, thus, would seem a rea-
sonable place to start to investigate if the current procedure would
produce a similar pattern of results to those obtained from a post
hoc analysis of IRTs (Shull et al., 2001). This would serve to
validate the current procedure for the examination of less well
established findings, but would also serve to bolster the findings of
Shull et al. (2001; Shull, 2004) in the light of the difficulties in
interpreting the IRT log survivor plots mentioned by Bowers et al.
(2008).
To this end, in this experiment, three groups of rats responded
on the current contingency, each group with a different random
interval (RI) schedule of reinforcement: RI 30s, RI 60s, or RI 180s.
If previous results were to be replicated, it would be expected that
overall response rates would increase as the mean interval value
decreased (i.e., as the rate of reinforcement increased). However,
this would be reflected in an increase in the number of burst-
initiation responses made during the session, rather than in the
number of within-burst responses being made.
Method
Subjects. Twenty-four male, experimentally na ve Lister
Hooded rats served in the present experiment. The subjects were
approximately 34 months old at the start of training and had a
free-feeding body-weight range of 355435g. The subjects were
maintained at 85% of their free-feeding body weight throughout
the study. The subjects were housed in groups of four, with water
constantly available in the home cage.
Apparatus. Four identical operant conditioning chambers
(Campden Instruments Ltd.) were used. Each chamber measured
23.5 cm 23.5 cm 20.cm (length width height). Each
chamber was housed in a light and sound-attenuating case, venti-
lated by a fan that provided background masking noise (65db[A]).
Each chamber had two levers, positioned either side of a centrally
located food hopper. The hopper was covered by a hinged clear
Perspex flap. There were jeweled houselights above each of the
levers. Reinforcement consisted of one 45-mg food pellet and was
delivered to the food hopper.
Procedure
All of the subjects received two 30-min sessions of magazine
training, during which the levers were retracted from the cham-
ber, and food was delivered according to a variable time (VT)
60s schedule. Following this training, all the rats were given
two 15-min sessions of lever-press training with a continuous
reinforcement schedule in operation, one session on each lever
(the other lever being retracted from the chamber). After lever-
press training, both of the levers were inserted into the chamber,
and the rats were given four 30-min sessions of training on a
concurrent RI 15s RI 15s schedule (i.e., the probability of
reinforcement becoming available for a response after each
second was 1/15). There was no change over delay in operation
to encourage alternation responding that would form part of the
response sequence to be studied at a later point in the experi-
ment. Following these four sessions, the rats experienced a
further six 60-min sessions of a concurrent RI 15s RI 15s
schedule, but this time the lights above the levers would both be
illuminated for 45s, during which time the schedule was in
operation, or would both not be illuminated for 75s, during
which no reinforcement was available. At the end of these six
sessions, the rats were making the majority of their response in
the presence of the light, and very few in its absence.
Following this training, the rats were then all exposed to the target
contingency, whose parameters were systematically altered over the
course of 10 further training sessions. In these sessions, initially the
light above the left lever was illuminated, and the light above the right
lever was not illuminated. A response to the left lever extinguished
the light above the left lever, and illuminated the light above the right
lever. While the light above the left lever was illuminated, a response
to the right lever had no programmed consequences. After illumina-
tion of the light above the right lever, responses to the right lever
could be reinforced according to a particular schedule. Failure to
respond to the right lever for a period of time while the light above
that lever was illuminated resulted in the extinction of the light above
the right lever, and responses to that lever having no programmed
consequences. The light above the left lever would then be illumi-
nated, and the sequence started again. The schedule continued to
operate irrespective of which light was illuminated. During these
sessions, an RI 15s schedule was in operation for all of the rats
(which could be satisfied by a response to the right lever only
when the right light was illuminated). During the first two ses-
sions, failure to make a response to the right lever for 10s while the
right light was illuminated resulted in the extinction of the right
lever. This value was systematically reduced by 1s every two
sessions, so that by the ninth and tenth sessions of this phase of the
training, the value was 6s.
Following this training regime, the rats were divided into three
groups of eight rats. One group of rats responded on an RI 30s
schedule, one group responded on an RI 60s schedule, and the final
group responded on an RI 180s schedule. The criterion value for all
groups for nonresponding extinguishing the light above the right lever
was 5s. All sessions lasted until the subjects had earned 30 reinforcers.
The subjects were exposed to these contingencies for 90 sessions (and
by the end of training around 70% of all responses emitted were to
illuminated levers; only these responses were analyzed).
3
RESPONSE RATE COMPONENTS
Results and Discussion
The data from the last six sessions of training, which were taken
to reflect steady-state performance, were analyzed. The mean
(standard deviation) obtained numbers of reinforcers per minute
for the three groups were: RI 30s, 1.6 (0.1); RI 60s, 0.8 (0.1); RI
180s, 0.4 (0.1). An analysis of variance (ANOVA) conducted on
these data revealed a statistically significant effect of group, F(2,
21) 324.19, p .001, and Tukeys Honestly Significant Dif-
ference (HSD) tests revealed that all pairwise comparisons be-
tween the groups were statistically significant, all ps .001. These
data show that the programming of the three schedules was oper-
ating broadly as planned.
The mean (standard deviation) overall responses per minute (all
responses to illuminated levers for the whole session period) for
the three groups were: RI 30s, 52.2 (16.0); RI 60s, 31.5 (6.1); RI
180s, 13.8 (5.4). These data were analyzed by an ANOVA that
revealed a statistically significant effect of group, F(2, 21)
27.44, p .001. Tukeys HSD tests revealed that all pairwise
comparisons between the groups were statistically significant, all
ps .01. This difference corroborates what has long been known;
that is, richer interval schedules of reinforcement produce higher
rates of response than leaner interval schedules of reinforcement.
In fact, the response rates obtained in this study were in approxi-
mate proportion to the obtained rates of reinforcement in accor-
dance with predictions based on the Matching Law (Herrnstein,
1970; although it might be noted that simple proportionality is not
the only prediction that could be derived from Herrnsteins equa-
tion when applied to a single manipulandum). This suggests that
the programming was producing the expected rates of response
overall for the rats.
The mean (standard deviation) number of burst-initiation re-
sponses made by the three groups per minute were: RI 30s, 11.1
(2.3); RI 60s, 9.1 (1.3); RI 180s, 6.3 (1.5). An ANOVA revealed
a statistically significant effect of group, F(2, 21) 15.27, p
.001, and Tukeys HSD tests revealed that the difference between
the RI 30s and the RI 180s groups, and that between the RI 60s and
the RI 180s groups, were statistically significant, ps .05. Of
course, these response rates include periods of time in which it
would be impossible to make a burst-initiating response. Recalcu-
lating the rate of burst-initiation responses after the time spent
responding in the bursts (including the 5-s periods without a
response) revealed a similar numerical pattern to that above, but
with much inflated rates of response (due to relatively large
portions, around 50%, of the session being taken up in 5-s periods
spent not responding). These recalculated mean (standard devia-
tion) responses per minute were: RI 30s, 107.1 (147.0); RI 60s,
35.2 (18.2); RI 180s, 14.0 (7.1). An ANOVA revealed only a
marginally statistically significant effect of group, F(2, 21) 2.60,
p .08, most likely due to the large variance in the groups. Given
this suggestion, an additional analysis, using an ANOVA con-
ducted on the burst initiation rate following a square root trans-
formation (as the variance will increase with the mean), revealed
a statistically significant difference between the groups, F(2, 21)
5.51, p .05. Tukeys HSD tests conducted on these transformed
data revealed a difference between the RI 30s group and each of
the other two groups. These differences partly corroborate what
has been apparent in the log survivor plots of IRT distributions
from single manipulandum situations, in that the number of burst-
initiating responses is determined by the rate of reinforcement
(see Shull et al., 2001; Shull, 2004).
The group mean (standard deviation) number of responses per
burst were: RI 30s, 3.9 (1.9); RI 60s, 2.5 (0.6); RI 180s, 1.2 (0.6).
An ANOVA revealed a statistically significant effect of group,
F(2, 21) 9.95, p .01, and Tukeys HSD tests revealed that
only the difference between the RI 30s and the RI 180s groups was
statistically significant. It should be noted that not all burst-
initiation responses were followed by responses to the within-
burst lever; approximately half of the burst-initiation responses
for each group were not followed by responses to the within-
burst lever (RI 30s, 53%; RI 60s, 45%; RI 180s, 47%). Calculat-
ing the numbers of responses per burst, in those bursts that pro-
duced a response, gives the following group mean (standard
deviation) results: RI 30s, 21.2 (15.9); RI 60s, 12.5 (11.7); RI 180s,
14.0 (14.0). An ANOVA conducted on these data revealed no
statistically significant effect of group, F 1. While there were
numerical differences between the groups, and some of these
differences were significant, on the whole, these data relating to
the number of responses per burst show little clear difference
between the groups. Such data corroborate what is apparent from
the post hoc inspection of IRT distributions in previous reports
(see Shull et al., 2001).
In addition, the group mean (standard deviation) rate of within-
burst responding for the three groups (from the first response to the
burst lever to the last, and excluding the 5s time of not respond-
ing at the end of the bursts) was calculated, and was: RI 30s, 247.9
(83.3); RI 60s, 142.3 (26.8); RI 180s, 61.8 (23.8). An ANOVA
revealed a statistically significant effect of group, F(2, 21)
25.35, p .001, and Tukeys HSD tests revealed that all pairwise
differences between the groups were statistically significant.
Taken together with the nonsignificant trends toward greater num-
bers of within-burst response noted above, these within-burst
response rate data suggest that there may be a modest relationship
between this within-burst responding variable and the rate of RI
reinforcement, as suggested in data presented by Shull et al.,
2004).
In summary, the present experiment replicated almost all fea-
tures of the log survivor plots produced previously, and the results
are consistent with the analysis of the effects on burst-initiating
responses, and within-burst responses produced by those reports.
This suggests that the current experimental procedure for investi-
gating these phenomena has criterion validity with other types of
post hoc analysis, and the results from additional studies using this
present procedure may well have validity in illuminating these
processes.
Experiment 2
The first experiment corroborated one prediction from the log
survivor analysis of IRT patterns, that on RI schedules, numbers of
burst-initiation responses vary directly with the rate of reinforce-
ment, but that within-burst responses are largely unaffected by
this manipulation (e.g., Shull et al., 2001; Shull, 2004). The second
study turns to examine the finding that, on random ratio (RR)
schedules, the value of the ratio does not impact the number of
burst-initiation responses, but that, as the ratio increase in size,
the number of within-burst responses increases (Bowers et al.,
2008; Shull et al., 2001; Shull & Grimes, 2003).
4
REED
To this end, three groups of rats were studied, each group
responding on a different RR schedule of reinforcement: RR-10,
RR-30, and RR-60. If previous results were to be replicated, it
would be expected that overall response rates would increase as
the ratio value increased (at least for this range of RR values, see
Reed & Hall, 1988). However, this would be reflected in an
increase in the number of within-burst responses, rather than in
the number of burst-initiation responses being made.
Method
Subjects and apparatus. Twenty-four male, experimentally
na ve Lister Hooded rats served in the present experiment. The
subjects were approximately 34 months old at the start of train-
ing, and had a free-feeding body-weight range of 380445g. The
subjects were housed and maintained as described in Experiment
1. The apparatus was that described in Experiment 1.
Procedure
The subjects were magazine and lever-press trained as described
in Experiment 1. After lever-press training, both of the levers were
inserted into the chamber, and the rats were given four 30-min
sessions of training on a concurrent RI 15s VI 15s schedule, as
described in Experiment 1. Following these four sessions, the rats
experienced a further six 60-min sessions of a concurrent RI 15s
RI 15s schedule, with both lights above the levers illuminated for
45s, during which time the schedule was in operation, or both not
illuminated for 75s, during which no reinforcement was available,
as described in Experiment 1. Following this exposure, the rats
were then all exposed to the training contingency, also as described
in Experiment 1, over the course of 10 further training sessions.
Following this training regime, the rats were divided into three
groups of eight rats. One group of rats then responded on a RR 10
schedule, one group responded on a RR 30 schedule, and the final
group responded on a RR 60 schedule (in all cases, the probability
of each response being reinforced was set at 1/RR value). The
criterion value for all groups for nonresponding extinguishing
the light above the right lever was 5s. All sessions lasted until the
subjects had earned 30 reinforcers. The subjects were exposed to
these contingencies for 90 sessions (as in Experiment 1, around
70% of all responses emitted were to illuminated levers, and these
were the only responses analyzed for this study).
Results and Discussion
The data from the last six sessions of training, which were taken
to reflect steady-state performance, were analyzed. The mean
(standard deviation) obtained numbers of reinforcers per minute
for the three groups were: RR-10, 6.5 (2.3); RR-30, 3.4 (1.3);
RR-60, 2.1 (0.8). An ANOVA conducted on these data revealed a
statistically significant effect of group, F(2, 21) 15.86, p .001,
and Tukeys Honestly Significant Difference (HSD) tests revealed
that the pairwise comparisons between the RR-10 and RR-30
groups, and between the RR-10 and RR-60 groups, were statisti-
cally significant, ps .01.
The mean (standard deviation) overall responses per minute (all
responses to illuminated levers for the total session) for the three
groups were: RR-10, 69.9 (27.6); RR-30, 101.7 (38.7); RR-60,
125.4 (43.5). An ANOVA that revealed a statistically significant
effect of group, F(2, 21) 4.52, p .05. Tukeys HSD tests
revealed that only the difference between the RR-10 and RR-60
groups was statistically significant, p .05.
The mean (standard deviation) number of burst-initiation re-
sponses made by the three groups per minute were: RR-10, 12.0
(4.9); RR-30, 6.1 (3.9); RR-60, 1.9 (0.8). An ANOVA revealed a
statistically significant effect of group, F(2, 21) 15.87, p .001.
Tukeys HSD tests revealed that the differences between the
RR-10 and RR-30 groups, and RR-10 and RR-60 groups, were
statistically significant, both ps .05. Of course, these response
rates include periods of time in which it would be impossible to
make a burst-initiating response. Recalculating the rate of burst-
initiation responses after the time spent responding in the bursts
(including the 5s periods without a response) revealed a similar
numerical pattern to that above. These recalculated mean (standard
deviation) responses per minute were: RR-10, 26.5 (18.4); RR-30,
11.9 (13.8); RR-60, 2.1 (0.9). An ANOVA revealed a statistically
significant effect of group, F(2, 21) 6.80, p .01. Tukeys HSD
tests revealed that the difference between the RR-10 and RR-60
groups was statistically significant, p .05.
The above data do not conform precisely to the findings from
the log survivor plots of IRT distributions from single manipulan-
dum situations, which suggest that the number of burst-initiating
responses on RR schedules is not greatly affected by the ratio
value (see Bowers et al., 2008; Shull et al., 2001; Shull & Grimes,
2003). However, it should be noted that these data would appear to
corroborate what was apparent from the analysis of the RI perfor-
mance in Experiment 1; that is, the rate of burst-initiation
responses varies with the rate of reinforcement (see also Shull et
al., 2001; Shull, 2004).
The group mean (standard deviation) number of responses per
burst were: RR-10, 5.0 (0.9); RR-30, 18.2 (5.6); RR-60, 68.6
(10.3). An ANOVA revealed a statistically significant effect of
group, F(2, 21) 196.02, p .001, and Tukeys HSD tests
revealed that all pairwise differences between the groups were
statistically significant, all ps .01. While it should be noted that
not all burst-initiation responses were followed by responses to
the within-burst lever; approximately a quarter of the burst-
initiation responses for each group were not followed by re-
sponses to the within-burst lever (RR-10, 24%; RR-30, 18%;
RR-60, 27%), correcting for this did not alter the pattern of results.
Calculating the numbers of responses per burst, in those bursts that
produced a response, gives the following group mean (standard
deviation) results: RR-10, 6.5 (1.1); RR-30, 22.2 (6.7); RR-60,
93.8 (6.5). An ANOVA revealed a statistically significant effect of
group, F(2, 21) 582.23, p .001, and Tukeys HSD tests
revealed that all pairwise differences between the groups were
statistically significant, all ps .01. It might be noted that in both
of these analyses, the RR-60 group appeared to have to emit
greater numbers of responses than might be expected to obtain
reinforcement according to that schedule, this can only be due to
random variations in the programming requirements. However,
what is apparent from both of these analyses, is that as the ratio
value increased, greater numbers of responses were emitted
within-burst, as has been suggested from IRT analyses (Bowers
et al., 2008; Shull et al., 2001).
The group mean (standard deviation) rate of within-burst re-
sponding for the three groups (from the first response to the
5
RESPONSE RATE COMPONENTS
burst lever to the last, and excluding the 5s time of not respond-
ing at the end of the bursts) reflected the above values, and was:
RR-10, 63.0 (5.5); RR-30, 121.1 (7.6); RR-60, 263.4 (25.8). An
ANOVA revealed a statistically significant effect of group, F(2,
21) 339.24, p .001, and Tukeys HSD tests revealed that all
pairwise differences between the groups were statistically signif-
icant. These within-burst rates of responding conform, generally,
to the rates of responding that might be expected in typical single-
manipulandum schedules for these ratio values (see Reed & Hall,
1988), with one exception; namely, that the rates of response
observed are numerically greater than in such procedures, a trend
that was especially noticeable in the RR-60 group (i.e., four
responses per second), and, in fact, this within-burst rate suggests
some unitization of responses had occurred (see Reed, Schacht-
man, & Hall, 1991).
In summary, the present experiment replicated several features
of the log survivor plots produced previously, and the results are
consistent with the analysis of the effects on burst-initiating
responses, and within-burst responses produced by those reports.
Experiment 3
The final study aimed to directly compare the impact of RI and
RR schedules on burst-initiation and within-burst responding
when the rate of reinforcement between the schedules is equated.
Thus far, only one study has attempted to make this comparison
(Bowers et al., 2008) using an IRT analysis. The results from this
latter study were made somewhat difficult to interpret by the lack
of clear points of inflection in the log survivor plots of the IRTs,
but in general, it was noted that RR schedules produced greater
numbers of within-burst responses, and RI schedules tended to
produce greater numbers of burst-initiation responses.
Should such a finding be replicated using the current procedure,
it would suggest a number of issues that need to be considered
when interpreting standard RR versus RI response rate differences
(e.g., Ferster & Skinner, 1957; Peele et al., 1984). Typically, the
higher response rate seen on a RR schedule has been the starting
point for an analysis of why the RR schedule maintains stronger
responding than a RI schedule (e.g., Baum, 1973; Morse, 1966;
Peele et al., 1984). Certainly, if there are greater numbers of
within burst responses on a RR schedule, compared to a RI
schedule, then these analyses would be supported for these within
burst responses. However, if there is a greater tendency to initiate
responding on a RI schedule than on a RR schedule, it may suggest
that strength of responding maintained by the schedules, may not
be so easily established by simple analysis of overall response
rates. Some evidence that seems to corroborate this suggestion
comes from a cross-experimental comparison of the RR-60 group
from Experiment 2, and the RI 30s group from Experiment 1.
These two groups had similar rates of reinforcement to one an-
other, which would allow some control of this variable in this
comparison. The comparison of these groups reveals that the RR
group had a higher rate of within-burst responding (as predicted
by previous studies, see Ferster & Skinner, 1957; Peele et al.,
1984), but the RI group had a higher rate of burst-initiation.
However, such cross-experimental comparisons are not ideal, and
a direct comparison needs to be made. To these ends, two groups
of rats were studied, one group responding on a RR schedule of
reinforcement, and the other on a yoked RI schedule of reinforce-
ment.
If previous results, and the above cross-experimental compari-
son, were to be replicated, it would be expected that overall
response rates would be higher on the RR schedule (e.g., Peele et
al., 1984), there would be greater numbers of within burst
responses on the RR schedule (Bowers et al., 2008), but greater
numbers of burst initiating responses on the RI schedule (cf.
Bowers et al., 2008; Nevin et al., 2001; Shull et al., 2001).
Method
Subjects and apparatus. Sixteen male, experimentally na ve
Lister Hooded rats served in the present experiment. The subjects
were approximately 45 months old at the start of training and had
a free-feeding body-weight range of 405480g. The subjects were
housed and maintained as described in Experiment 1. The appa-
ratus was that described in Experiment 1.
Procedure. The subjects were magazine and lever-press
trained as described in Experiment 1. After lever-press training,
both of the levers were inserted into the chamber, and the rats were
given four 30-min sessions of training on a concurrent RI 15s RI
15s schedule, as described in Experiment 1. Following these four
sessions, the rats experienced a further six 60-min sessions of a
concurrent RI 15s RI 15s schedule, with both lights above the
levers illuminated for 45s, during which time the schedule was in
operation, or both not illuminated for 75s, during which no rein-
forcement was available, as described in Experiment 1. The rats
were then all exposed to the contingency as described in Experi-
ment 1 for 10 further training sessions.
Following this training regime, the rats were divided into two
groups of eight rats. One group of rats then responded on a RR 30
schedule, and the other group responded on a yoked RI schedule.
The RR and RI schedules were programmed as described above in
Experiments 1 and 2. The rats were placed in master-yoked animal
pairs, which remained constant throughout the study. The times
between the successive reinforcers obtained by the master RR rat,
became the successive criterion interval times for the yoked RI rat.
The criterion value for all groups for nonresponding extinguishing
the light above the right lever was 5s. All sessions lasted until the
subjects had earned 30 reinforcers. The subjects were exposed to
these contingencies for 90 sessions.
Results and Discussion
The data from the last six sessions of training, which were taken
to reflect steady-state performance, were analyzed. The mean
(standard deviation) obtained numbers of reinforcers per minute
for the groups was: RR-30, 3.7 (0.7); RI-y, 3.4 (0.6). A t test
revealed no statistically significant effect of group, t 1, suggest-
ing that the yoking procedure had worked as expected.
The mean (standard deviation) overall responses per minute (all
responses to both levers for the total session) for the groups were:
RR-30, 118.2 (23.5); RI-y, 65.5 (21.3). A t test revealed a statis-
tically significant effect of group, t(14) 4.74, p .001. This
effect replicates the standard RR versus RI response rate difference
found in multiple studies, previously (e.g., Ferster & Skinner,
1957; Peele et al., 1984; Zuriff, 1970).
However, the mean (standard deviation) number of burst-
initiation responses made by the groups per minute showed the
6
REED
reverse pattern of results, with the group means being: RR-30, 5.8
(0.9); RI-y, 11.3 (1.6), t(14) 8.77, p .001. Of course, these
response rates include periods of time in which it would be
impossible to make a burst-initiating response. Recalculating the
rate of burst-initiation responses after the time spent responding
in the bursts (including the 5-s periods without a response) re-
vealed a similar numerical pattern to that above, but with much
inflated rates of response and variability. These recalculated mean
(standard deviation) responses per minute were: RR-30, 76.9
(81.7); RI-y, 230.9 (194.4), t(14) 2.07, p .05. These data
corroborate cross-experimental comparisons from the study of IRT
log survivor plots (cf. Bowers et al., 2008; Shull et al., 2001), and
also suggestions from previous reports that, when the rate of
responding is equated across RR and RI schedules, that subjects
will differentially respond to access the RI rather than the RR
schedule (cf. Nevin et al., 2001).
The group mean (standard deviation) number of responses per
burst were: RR-30, 19.2 (3.6); RI-y, 4.8 (1.9). A t test revealed a
statistically significant effect of group, t(14) 10.11, p .001. As
not all burst-initiation responses were followed by responses to
the within-burst lever (RR-30, 21%; RI-y, 28%), correcting for
this did not alter the pattern of results. Calculating the numbers of
responses per burst, in those bursts that produced a response, gives
the following group mean (standard deviation) results: RR-30,
24.5 (5.3); RI-y, 6.6 (2.3), t(14) 8.70, p .001. In both of these
analyses, the RR-30 group emitted greater numbers of within-
burst responses than a yoked RI group. The group mean (standard
deviation) rate of within-burst responding for the three groups
(from the first response to the burst lever to the last, and
excluding the 5-s time of not responding at the end of the bursts)
reflected the above values, and was: RR-30, 151.6 (40.7); RI-y,
100.2 (45.1), t(14) 2.39, p .05.
In summary, the present experiment confirmed what was pre-
dicted on the basis of the previous literature from the log survivor
plots: there was a standard RR versus RI difference in overall
response rate (see Ferster & Skinner, 1957), which appeared to be
due to a greater number of responses being emitted once respond-
ing was initiated (see Bowers et al., 2008; Shull et al., 2001).
However, there was a greater tendency to initiate responding on
the RI relative to the RR schedule, despite an equation of the
overall rates of reinforcement. These findings were more pro-
nounced than those obtained from the cross-experimental compar-
ison of Experiment 1 and 2, noted above, but this may reflect
somewhat different rates of reinforcement obtained, or even a
difference between the effects of single and multiple schedules.
Overall, these findings suggests that previous indications of greater
response strength for RI than RR schedules: for example, subjects
will chose RI schedules in preference to RR schedules, at least
under some conditions (e.g., Williams, 1985), and show greater
resistance to change following RI than RR schedules (Nevin et al.,
2001), may well be indexing the same aspect of behavior as the
current burst initiation measure.
General Discussion
The current series of studies were designed to explore a novel
approach to analyzing the different components of response rate
that are produced by exposure to free-operant schedules of rein-
forcement. It has been suggested that overall response rate com-
prises a tendency to initiate responding and to continue to respond
once the bout is initiated. Previous post hoc analyses of IRT data
have suggested several features of these different aspects of re-
sponding that the current experimental procedure broadly con-
firmed. That is, increasing the size of a RI schedule decreases the
number of burst-initiation responses, but has less effect on
responding once the burst has been initiated (Experiment 1); that
the major difference between a RR schedule and a RI schedule, is
not in the number of burst-initiation responses, but in the number
of shorter within-burst responses that are emitted, with greater
numbers of within-burst responses being emitted on a RR sched-
ule (Experiments 2 and 3).
These results have lead to the claim that burst-initiation
responses are controlled by motivational factors (e.g., rate of
reinforcement), and the within-burst responses are controlled
by schedule factors (e.g., reinforcement of particular IRTs).
This suggestion mirrors the distinction between the strengthen-
ing and shaping properties of reinforcement made by Morse
(1966; see Shull et al., 2001). The modulation of within burst
responding seen in the current studies (notably Experiments 2
and 3), supports this type of analysis for this component of
responding. Alternatively, Nevin (1979; Grace & Nevin, 1997)
have suggested that resistance to change is a stronger index of
strength of learning than response rate. When directly compared
on this measure, RI schedules demonstrate a greater resistance
to change than RR schedules, suggesting stronger learning on
the RI, than the RR, schedule (Nevin et al., 2001). That subjects
in Experiment 3 were more likely to initiate responding on a RI
schedule than a RR schedule is in line with this suggestion, and
also in line with some results from choice studies (see Williams,
1985; but note that this may only occur under some parameters,
and further exploration of this effect would be wise before firm
conclusions are drawn).
Although the primary focus of the current series of studies was
empirical, and an attempt to develop an experimental procedure
that could corroborate, or otherwise, the conclusions based on a
more analytic approach to schedule behavior, the results do have
some implications for theories of schedule performance. In partic-
ular, they suggest that the traditional approach of attempting to
accommodate all responses with a single molecular or molar view
may be misguided. Rather, it may well be that, not only do these
two aspects of the environment control behavior, more or less
strongly, under different conditions (see Reed, 2007), but that
different aspects of behavior may come under different aspects of
environmental control (burst-initiations being more sensitive to
molar contingencies, and within-burst responses to molecular pro-
cesses). Such a set of functions will, of course, make analysis of
schedule performance more complex, but may offer opportunities
to unify the various competing views.
It should be noted that, in the current series of experiments, an
arbitrary feature of the design was the selection of a 5-s pause from
responding as the criterion for assuming that a response-burst had
been terminated. The use of this particular length of time was
indicated by an analysis of prior data on rats responding patterns
on schedules of reinforcement, which had suggested that this value
was appropriate (e.g., Bowers et al., 2008; Shull & Grimes, 2003).
Nevertheless, it is wise to speculate, briefly, about the implications
of choosing a value shorter, or longer, than the current 5s, as it is
possible that different results would have been generated with the
7
RESPONSE RATE COMPONENTS
use of a different criterion cut off point. The selection of a longer
cut-off criterion would have tended to produce fewer response
bursts containing greater numbers of responses, and this could
have, potentially, resulted in fewer burst-initiation responses.
Presumably, the greater the rats tendency to respond within-
burst, the more pronounced this predicted pattern of data would
have been (i.e., the less likely it would be that the longer cut-off
criterion would be reached). In turn, this tendency might have
interacted with the above general truism concerning longer crite-
rion cut-off times producing fewer bursts. Hence, given the results
from Experiments 1 and 2, longer ratios and shorter intervals may
have lead to fewer bursts, and fewer burst-initiations, which would
have made the pattern of differences between the schedules that
were noted in the current studies even more pronounced. The
opposite is true, of course, if a shorter cut-off criterion had been
used. In the absence of further data (or the ability to analyze the
current data more precisely), this issue of whether the overall
pattern of data remained the same with different cuts is one for
future empirical investigation.
In addition to establishing some of these effects, the current
report also aimed to document an experimental procedure that
might be useful in this area. In this way it was hoped to offer a
more experimental, than analytic, approach to this area, based on
the notion of a revealed operant (Mechner, 1992; Pear & Rector,
1979). Such a tool could be a useful additional approach to
compliment the post hoc analysis of IRT distributions discussed
above. A number of problems with the latter approach were
outlined in the General Introduction, including the method of
identifying within-burst and between-burst responses that it
depends upon post hoc inspection of the IRTs produced by the
organism. That the present experiments replicated almost all fea-
tures of the log survivor plots produced previously, and the results
are consistent with the analysis of the effects on burst-initiating
responses, and within-burst responses produced by those reports,
suggests that the current experimental procedure for investigating
these phenomena has criterion validity with other types of post hoc
analysis, and the results from additional studies using this present
procedure may well have validity in illuminating these processes,
without incurring the problems of interpretation inherent in the pot
hoc analyses.
References
Bacha-Mendez, G., Reid, A. K., & Mendoza-Soylovna, A. (2007). Resur-
gence of integrated behavioral units. Journal of the Experimental Anal-
ysis of Behavior, 87, 524.
Baum, W. M. (1973). The correlation-based law of effect. Journal of the
Experimental Analysis of Behavior, 20, 137153.
Baum, W. M. (1993). Performances on ratio and interval schedules of
reinforcement: Data and theory. Journal of the Experimental Analysis of
Behavior, 59, 245264.
Bennett, J. A., Hughes, C. E., & Pitts, R. C. (2007). Effects of metham-
phetamine on response rate: A microstructural analysis. Behavioural
Processes, 75, 199205.
Blough, D. S. (1963). Interresponse time as a function of continuous
variables: A new method and some data. Journal of the Experimental
Analysis of Behavior, 6, 237246.
Bowers, M. T., Hill, J., & Palya, W. L. (2008). Interresponse time struc-
tures in variable-ratio and variable-interval schedules. Journal of the
Experimental Analysis of Behavior, 90, 345362.
Cole, M. R. (1999). Molar and molecular control in variable-interval and
variable-ratio schedules. Journal of the Experimental Analysis of behav-
ior, 71, 319328.
Dack, C., McHugh, L., & Reed, P. (2009). Generalization of causal
efficacy judgments after evaluative learning. Learning and Behavior.
Dickinson, A., Peters, R. C., & Shechter, S. (1984). Overshadowing of
responding on ratio and interval schedules by an independent predictor
of reinforcement. Behavioural Processes, 9, 421429.
Ferster, C. B., & Skinner, B. F. (1953). Schedules of reinforcement.
Appleton-Century-Crofts: New York.
Grace, R. C., & Nevin, J. A. (1997). On the relation between preference
and resistance to change. Journal of the Experimental Analysis of Be-
havior, 67, 4365.
Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental
Analysis of Behavior, 13, 243266.
Kessel, R., & Lucke, R. (2008). An analytic form for the response rate
analysis of Shull, Gaynor, and Grimes with applications and extensions.
Journal of the Experimental Analysis of Behavior, 90, 363386.
Kulubekova, S., & McDowell, J. J. (2008). A computational model of
selection by consequences: Log survivor plots. Behavioural Processes,
78, 291296.
Lattal, K. A., & Neef, N. A. (1996). Recent reinforcement-schedule re-
search and applied behavior analysis. Journal of Applied Behavior
Analysis, 29, 213230.
McDowell, J. J., & Wixted, J. T. (1986). Variable-ratio schedules as
variable-interval schedules with linear feedback loops. Journal of the
Experimental Analysis of Behavior, 46, 315329.
Mechner, F. (1992). The revealed operant: A way to study the character-
istics of individual occurrences of operant responses. Cambridge, MA:
Cambridge Center for Behavioral Studies.
Mellgren, R. L., & Elsmore, T. F. (1991). Extinction of operant behavior:
An analysis based of foraging considerations. Animal Learning & Be-
havior, 19, 317325.
Morse, W. H. (1966). Intermittent reinforcement. In W. K. Honig (Ed.),
Operant behavior: Areas of research and application (pp. 52109). New
York: Appleton Century-Crofts.
Nevin, J. A. (1979). Reinforcement schedules and response strength. In
M. D. Zeiler & P. Harzem (Eds.), Reinforcement and the organization of
behavior. Chichester, England: Wiley.
Nevin, J. A., Grace, R. C., Holland, S., & McLean, A. P. (2001). Variable-
ratio versus variable-interval schedules: Response rate, resistance to
change, and preference. Journal of the Experimental Analysis of Behav-
ior, 76, 4374.
Pear, J. J., & Rector, B. L. (1979). Constituents of response rate. Journal
of the Experimental Analysis of Behavior, 32, 341362.
Peele, D. B., Casey, J., & Silberburg, A. (1984). primacy of
interresponse-time reinforcement in accounting for rate differences
under variable-ratio and variable-interval schedules. Journal of Ex-
perimental Psychology: Animal Behaviour Processes, 10, 149167.
Raia, C. P., Shillingford, S. W., Miller, H. L., Jr., & Baier, P. S. (2000).
Interaction of procedural factors in human performance on yoked
schedules. Journal of the Experimental Analysis of Behavior, 74,
265281.
Reed, P. (2007). The role of response rate in determining sensitivity to
the molar feedback function relating response rate and reinforcement
rate on VI schedules of reinforcement. Journal of Experimental
Psychology: Animal Behavior Processes, 33, 428439.
Reed, P., & Hall, G. (1988). The schedule dependency of the signalled
reinforcement effect. Learning and Motivation, 19, 387407.
Reed, P., & Morgan, T. A. (2006). Resurgence of response-sequence
during extinction in rats shows a primacy effects. Journal of the Exper-
imental Analysis of Behavior, 86, 307315.
Reed, P., Schachtman, T. R., & Hall, G. (1988). Overshadowing and
8
REED
potentiation of instrumental responding in rats as a function of the
schedule of reinforcement. Learning and Motivation, 19, 1330.
Reed, P., Schachtman, T. R., & Hall, G. (1991). The effect of signalled
reinforcement on the formation of behavioral units. Journal of Experi-
mental Psychology: Animal Behavior Processes, 17, 475485.
Shull, R. L. (2004). Bouts of responding on variable interval schedules:
Effects of deprivation level. Journal of the Experimental Analysis of
Behavior, 81, 155167.
Shull, R. L., Gaynor, S. T., & Grimes, J. A. (2001). Response rate
viewed as engagement bouts: Effects of relative reinforcement and
schedule type. Journal of the Experimental Analysis of Behavior, 75,
247274.
Shull, R. L., Gaynor, S. T., & Grimes, J. A. (2002). Response rate viewed
as engagement bouts: Resistance to extinction. Journal of the Experi-
mental Analysis of Behavior, 77, 211231.
Shull, R. L., & Grimes, J. A. (2003). Bouts of responding from variable-
interval reinforcement of lever pressing by rats. Journal of the Experi-
mental Analysis of Behavior, 80, 159171.
Shull, R. L., Grimes, J. A., & Bennett, J. A. (2004). Bouts of responding:
The relation between bout rate and the rate of variable-interval rein-
forcement. Journal of the Experimental Analysis of Behavior, 81, 6583.
Williams, B. A. (1985). Choice behavior in discrete-trial concurrent VI-VR:
A test of maximizing theories of matching. Learning & Motivation, 16,
423443.
Zuriff, G. E. (1970). A comparison of variable-ratio and variable-interval
schedules of reinforcement. Journal of the Experimental Analysis of
Behavior, 13, 369374.
Received April 11, 2009
Revision received February 17, 2010
Accepted February 19, 2010
Online First Publication
APA-published journal articles are now available Online First in the PsycARTICLES database.
Electronic versions of journal articles will be accessible prior to the print publication, expediting
access to the latest peer-reviewed research.
All PsycARTICLES institutional customers, individual APA PsycNET database package sub-
scribers, and individual journal subscribers may now search these records as an added benefit.
Online First Publication (OFP) records can be released within as little as 30 days of acceptance and
transfer into production, and are marked to indicate the posting status, allowing researchers to
quickly and easily discover the latest literature. OFP articles will be the version of record; the
articles have gone through the full production cycle except for assignment to an issue and
pagination. After a journal issues print publication, OFP records will be replaced with the final
published article to reflect the final status and bibliographic information.
9
RESPONSE RATE COMPONENTS