Evolution

The changes in behaviour

that accumulate across
generations are stored in
the genes

Learning
The changes in behaviour
that accumulate over a
lifetime are stored in the
central nervous system.

Two Types of Learning

Classical conditioning

Operant conditioning

Ivan Pavlov’s
Before Learning
“Psychic Secretions”
Meat Powder Salivation

Unconditioned Unconditioned
Ivan Pavlov
Stimulus (UCS) Response (UCR)
1849-1936

1
 Before Learning During Learning

Bell No response

Neutral
Stimulus Repeatedly pair
UCS and Neutral
Stimulus

After Learning
The term ‘psychic secretions’ was
Bell Salivation eventually replaced with the term,
conditioning, and eventually became
known as Classical Conditioning or
Pavlovian Conditioning.

Conditioned Conditioned
Stimulus (CS) Response (CR)

Test Yourself What about Pavlov’s Cat?

Every time Robert’s wife flushes the toilet when he’s in the
shower, the water becomes painfully hot. One day, as
Robert is stepping into the shower, he hears a flushing sound
and flinches. This flinch is an example of a(n):

A. neutral stimulus
B. unconditioned stimulus
C. unconditioned response
D. conditioned stimulus
E. conditioned response Pavlov’s dog, cat and
Cousin Earl

2
 John Watson and Behaviorism Black Box Psychology
• “Psychology should be a purely objective
experimental branch of natural science”
• believed that psychology had failed as a
science
• disliked introspection and hypothetical THE
concepts (such as “decisions”, “knowledge”, Stimulus  MIND  Response
“perception”, “consciousness”)
Do NOT Open

John B. Watson
1878-1958

Little Albert
Little Albert’s fear of the white rat
generalizes to the furry rabbit

video video

Whatever happened to Watson?

“Give me a dozen healthy infants, well-formed,
“I began to learn that it can be just as thrilling to
and my own specified world to bring them up in
watch the growth of a sales curve of a new
and I’ll guarantee to take any one at random
product as to watch the learning curve of
and train him to become any type of specialist I
animals and men.”
might select -- doctor, lawyer, artist, merchant,
chief, and yes, even beggar-man and thief, -- J. B. Watson, 1936
regardless of his talents, penchants,
tendencies, abilities, vocations, and the race of
his ancestors.”
-- J. B. Watson, 1924

3
 Classical Conditioning Concepts Does Learning Generalize?
• Generalization and discrimination “He who is bitten by the snake fears the lizard.”
• Higher-order conditioning -- Bugandan proverb
• Extinction
• Spontaneous recovery In fact, the phenomenon of
• Practical applications generalization was one of the
– aversive conditioning
– compensatory reaction hypothesis
mechanisms that Pavlov hoped
would permit conditioning to explain
all kinds of complex behaviour.

20 Generalization 20 Discrimination
What will happen if you
Generalization only give food after a
Drops of saliva

Drops of saliva
10,000 Hz tone but not an
gradient 8,000 or 12,000 Hz tone?

10 10

CS used in
training

0 0
6000 8000 10,000 12,000 14,000 6000 8000 10,000 12,000 14,000

Frequency of tuning fork in Hz Frequency of tuning fork in Hz

Higher-Order Conditioning Once an organism is conditioned, is it destined

always to show a conditioned response
whenever the conditioned stimulus is
presented?
Will Pavlov’s dog always salivate to bells?

The answer is no.

If the CS is presented enough times without
the UCS, then the CR will eventually
weaken and not occur.

4
 Extinction and Recovery Counterconditioning

• A Clockwork Orange • Antabuse

Compensatory-Reaction Hypothesis Compensatory-Reaction Hypothesis

NEW USER HABITUAL USER HABITUAL USER

New Location Familiar Location Familiar Location
Small Dose Small Dose Large Dose
(e.g., blood sugar levels in diabetic)

Why didn’t Vietnam vets crave heroin Why do diabetics blood sugar levels Drug
when they returned to the U.S.? go UP just before they use insulin? Drug Drug Drug Drug Drug
NOT
Administered Expected Administered Expected Administered
Expected
Drug Effect

Time

How is it that some drug addicts can overdose Compensatory Reaction Tolerance
when they’ve only taken their typical dosage? • can lead to cravings • need more drug to get same effect

Compensatory-Reaction Hypothesis
HABITUAL USER
New Location
Large Dose
(e.g., blood sugar levels in diabetic)

Drug
Drug
NOT
Administered
Drug Effect

Expected

Overdose
• without compensatory
reaction, drug is too strong

5
 Thorndike’s Puzzle Box Thorndike’s Cat’s Learning Curve

Edward Thorndike
(1874-1949)

“Trial and error learning”

First trial Later trial

Exploring Lever-pressing
Sniffing Exploring
Grooming Grooming
Tendency Tendency
Reaching Reaching
to perform to perform
Scratching Scratching
Reaching with paw Reaching with paw
Lever-Pressing Sniffing

Reward

Thorndike’s Law of Effect B. F. Skinner and the Skinner Box

Responses that produce a satisfying effect

in a particular situation become more likely
to occur again in that situation.
magazine
delivers food bar to
Responses that produce a discomforting B.F. Skinner press
effect become less likely to occur again in 1904 -1990

that situation.
lights
to
peck
video

6
Shaping Behaviour
Skinner believed that almost any behaviour
could be ‘shaped’ by operant methods.
Pigeons, for example, could be trained to
walk in circles, play ping-pong, or even play
a tune on a small piano, if successive
approximations to the desired behavior
were reinforced.
Skinner believed the same idea could be
used to explain how children learned to talk.
Shaping
Reward animal for successive approximations of
desired behavior
1) reward rat for approaching lever
2) reward rat for facing lever
3) reward rat for touching lever
4) reward rat for pressing lever
• Reward must promptly follow behavior
• Can get superstitious behavior if not careful

Operant Conditioning
Animals can • The animal is reinforced for certain operations on
develop astounding the environment
skills through • also called Instrumental or Skinnerian Conditioning
shaping

Reinforcement
• an outcome that affects the likelihood of an
operant response re-occuring

Extinction Response Curve

• Decline in operant responses with cessation
of reinforcement

Partial Reinforcement
• Response is not rewarded on every trial
• Makes behavior more resistant to extinction
• Can get extensive efforts for small rewards

7
 Partial Reinforcement Schedules
1. Fixed ratio: reinforced after a fixed
number of responses.
2. Variable ratio: reinforced after a
variable number of responses.
3. Fixed Interval: reinforced after a given
amount of time has elapsed.
4. Variable Interval: reinforced after a
variable interval of time has elapsed.
Reinforcement Schedules
Types of Reinforcement
Reinforcement increases behavior
• positive uses addition of stimulus
• negative uses removal of stimulus
Punishment decreases behavior
• positive uses addition of stimulus
• negative uses removal of stimulus
Negative reinforcement ≠ punishment!!!
Try these…
• Classify the following situations based on the
reinforcement schedule:
– You get 1 air mile for every $20 you spend at Shell
– You may win a large amount playing video poker
– You study for the tests that your professor gives you once per month
– You study regularly because your professor periodically gives “pop”
quizzes
– You get one Coca-cola for every $1 of change you put in the vending
machine
– You check your snail mail which comes once per day
– Your boss pays you $10/hour
– Your boss pays you 5 cents for every letter you engrave
– Your boss pays you a commission for each sale you make
Why do people gamble?
Try these…
Classify the following…
• If you bring your umbrella, you won’t get rained on
• A parent tells a child they can watch TV if they finish their
homework
• If you get out of bed, your roommate will stop yelling at you
to get up
• A police officer gives a speeding motorist a ticket
• A parent tells a child that if keeps acting up, he will get a
“time out”
• A dog gets a Milkbone if he rolls over and plays dead
• If you take an aspirin, your headache pain will go away
• If you get caught drunk driving, you will lose your license
8
 Differences Between Classical and
Operant Conditioning.
1. Timing of events
In classical conditioning the delivery of the
reinforcement (UCS) is controlled by the
experimenter.
In operant conditioning the delivery of the
reinforcement is controlled by the animal or
human subject.
2. Nature of responses that can be
trained
In classical conditioning the responses are
ones that are generally referred to as
“involuntary” (eye blinks, heart rate, etc.)
In operant conditioning, the responses are
ones that are generally described as
“voluntary” (pecking, hand movements,
speech, etc).

Evolutionary Perspectives “The Sauce Bearnaise Syndrome”

• The strong behaviorist view: Anything can be
conditioned to anything else (classically) and
any behavior can be trained (operantly) In which our hero
• Is this really true? develops a powerful
• Why is it easier to train pigeons to peck and aversion to Sauce
rats to bar press?
Bearnaise*

*a tarragon-
flavoured white sauce

Martin goes out to dinner with his wife, and they

have a lovely meal -- filet mignon with Sauce Is this a case of Classical Conditioning?
Bearnaise and asparagus.

They then go to see Tristan und Isolde (an opera

by Richard Wagner). Sauce
Bearnaise
In the middle of Act II, Martin becomes violently ill
and they have to go home. CS CR
He later finds out that he has the 24 hr stomach flu
(as do three of his co-workers). Illness Nausea
Ever since then, Martin cannot abide the taste or UCS UCR
smell of Sauce Bearnaise.

9
 Problems with this explanation Bright, Noisy, Tasty Water Expts.
What about the long gap between the CS • Are organisms evolutionarily
and the UCS? prepared to make particular
associations?
Why is food the CS? • The “Bright, Noisy, Tasty Water”
Why not Tristan und Isolde or
Experiments
Wagnerian opera? John Garcia

Why not the plate on which the food

was served?

Why not his wife?

Animals come prepared to

learn some things but not
Other food aversion effects
others. • unlike most other types of classical conditioning, the time window for
food aversion learning can be long (hours, days)
Rats will learn to associate • foods with a distinctive flavor are more likely to be associated with an
illness (caused by x-rays) illness
with taste, but not visual or
• novel foods are more likely to be associated with an illness
auditory information
• different animals have different preparedness for food aversion learning
Rats will learn to associate
pain with visual or auditory
cues, but not with taste.

Many examples of preparedness to learn

Song-learning in birds

Maze learning in rats

Route finding in honeybees

Language acquisition in humans

10