Human associative learning Lecture 2 Intro to human associative learning pt 2 Prof Mike Le Pelley [email protected] PSYC 2081 The Hats of Power • Post thoughts, insights, jokes, memes,
observations etc to the Moodle forum • Best entries will win one of four Hats of Power Summary of Lecture 1 • Contingency: How strongly are events related? • Learning is influenced by contingency (as you would hope!) • But that’s not the whole story – for example – Learning is gradual even though ΔP is constant; “learning curves” – When we learn about multiple cues at the same time, we don’t learn
about each one individually based on its contingency – Instead cues compete with each other Blocking as an example of “cue competition” • Learning is not the same as calculating contingency. So what
are people doing when they learn? • We can understand learning as the gradual formation of an
association (a link) between representations An alternative view: Association formation CS cue US outcome • What determines whether (and how fast) this association
strengthens? • Proposed by Rescorla & Wagner (1972) • Surprise is the key to learning – We only learn when something happens that we don’t expect We learn when we are surprised by something – If everything occurs as expected, we don’t learn anything new We don’t learn if we are not surprised • Rescorla & Wagner captured this idea in an equation, providing a
simple but powerful model of the learning process The Rescorla-Wagner model Robert
Rescorla Allan
Wagner ∆V
=
α β
( λ
– ΣV ) Salience
of cue Salience
of
outcome Observed
magnitude
of outcome Expected
magnitude
of outcome Σ means “sum” So ΣV means “summed
associative strength of all cues” ∆ means “change” So ∆V means “change in
associative strength” i.e., learning cue outcome V V = predictive Value (or associative strength) Amount of
learning The Rescorla-Wagner model ∆V
=
α β
( λ
– ΣV ) Prediction error: “Surprisingness” of
outcome • When prediction error is zero, there is no learning • Learning
=
Reduction in PE • Learning
=
Things becoming more expected, and less surprising Salience
of cue Salience
of
outcome Observed
magnitude
of outcome Expected
magnitude
of outcome Amount of
learning The Rescorla-Wagner model cue outcome V V = predictive Value (or associative strength) V = 0.4 ∆V
=
α
β
( λ
– ΣV ) Associative
strength Salience
of cue Salience of
outcome Observed
magnitude of
outcome Expected
magnitude of
outcome ∆Vvege =
0.5
x
0.8
( 1 – 0 )
=
0.4 Vvege =
0 Vvege =
0.4 0.5 0.8 1Let’s assume Vege illness Large
prediction
error V = 0.64 ∆V
=
α
β
( λ
– ΣV ) ∆Vvege =
0.5
x
0.8
( 1 – 0.4 )
=
0.24 Vvege =
0.4 Vvege =
0.64 Vege illness Smaller
prediction
error Associative
strength Salience
of cue Salience of
outcome Observed
magnitude of
outcome Expected
magnitude of
outcome 0.5 0.8 1Let’s assume Trial 0:
Vvege =
0 Trial 1:
Vvege =
0.40 Trial 2:
Vvege =
0.64 Trial 3:
Vvege =
0.78 Trial 4:
Vvege =
0.87 Trial 5:
Vvege =
0.92 0 0.5 1 0 5 10 15 As so cia tiv e
st re ng th Trial ∆V
=
α
β
( λ
– ΣV ) SURPRISING NOT SURPRISING Associative
strength Salience
of cue Salience of
outcome Observed
magnitude of
outcome Expected
magnitude of
outcome 0.5 0.8 1Let’s assume V = 0.99 Vege illness 0 0.5 1 0 5 10 15 As so cia tiv e
st re ng th Trial ∆V
=
α
β
( λ
– ΣV ) Associative
strength Salience
of cue Salience of
outcome Observed
magnitude of
outcome Expected
magnitude of
outcome ∆Vvege =
0.5
x
0.8
( 0 – 0.99 )
=
–0.40 Vvege =
0.99 Vvege =
0.59 0.5 0.8 0Let’s assume Negative prediction
error V = 0.59 Vege illness V = 0.35 ∆V
=
α
β
( λ
– ΣV ) ∆Vvege =
0.5
x
0.8
( 0 – 0.59 )
=
–0.24 Vvege =
0.59 Vvege =
0.35 Vege illness Smaller negative prediction
error Associative
strength Salience
of cue Salience of
outcome Observed
magnitude of
outcome Expected
magnitude of
outcome 0.5 0.8 0Let’s assume 00.5 1 0 5 10 15 20 25 As so cia tiv e
st re ng th Trial Acquisition Extinction Acquisition and extinction SURPRISING NOT SURPRISING NOT SURPRISINGSURPRISING ∆V
=
α
β
( λ
– ΣV ) Salience
of cue Salience of
outcome Observed
magnitude
of outcome Expected
magnitude
of outcome 0 0.5 1 0 5 10 15 As so cia tiv e
st re ng th Trial Associative
strength α = 0.5, β = 0.8 α = 0.5, β = 0.2 α = 0.2, β = 0.8 -80 -60 -40 -20 0 20 40 60 80 0 5 10 15 20 25 30 35 40 Ju dg m en t r at in g Trial Lopez & Shanks (unpublished) See Shanks 1995, ch2, p32 Rescorla-Wagner and contingency People • Learning is gradual, even though contingency (ΔP) does not change – “Learning curves” • The Rescorla-Wagner model also shows this pattern ∆P = 0.5 ∆P = -0.5 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0 5 10 15 20 25 30 35 40 As so cia tiv e
st re ng th Trial RW ∆P = 0.5 ∆P = -0.5 01 2 3 4 Bread Dates Ju dg m en t r at in g Aitken, Larkin & Dickinson (2000, Expt 1) Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? Rescorla-Wagner and blocking • Previous learning about apples blocks subsequent learning about bread • Cues are not learned about
independently • Instead, cues compete with each other 00.5 1 0 5 10 15 20 As so cia tiv e
st re ng th Trial ∆Vbread :
(λ – ΣV) Vapple Rescorla-Wagner and blocking Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? 00.5 1 0 5 10 15 20 As so cia tiv e
st re ng th Trial ∆Vbread :
(1 – ΣV) Vapple Rescorla-Wagner and blocking Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? 00.5 1 0 5 10 15 20 As so cia tiv e
st re ng th Trial ∆Vbread :
(1 – [Vapple + Vbread]) Vapple Rescorla-Wagner and blocking Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? 00.5 1 0 5 10 15 20 As so cia tiv e
st re ng th Trial Vapple Rescorla-Wagner and blocking Outcome not
surprising ∆Vbread :
(1 – [0.85 + Vbread]) Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? 00.5 1 0 5 10 15 20 As so cia tiv e
st re ng th Trial Vbread Vapple Outcome not
surprising Rescorla-Wagner and blocking = 0.15 Small
prediction
error ∆Vbread :
(1 – [0.85 + 0]) Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? Allergy Apples Bread Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? Rescorla-Wagner and blocking Allergy Carrot Dates Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? Rescorla-Wagner and blocking 01 2 3 4 Bread Dates Ju dg m en t r at in g Aitken, Larkin & Dickinson (2000, Expt 1) Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? • People learn a
weaker
association for
bread than dates • And so does the
RW model! 0 0.2 0.4 0.6 0.8 Bread Dates As so cia tiv e
st re ng th People RW Rescorla-Wagner and blocking Summary so far • Lecture 1:
Learning
=
Calculating contingency? • A different view:
Learning
=
Forming association between
representations • The Rescorla-Wagner model – Surprise drives association formation (learning) – RW is one model of association formation – there are many others RW is probably the most influential model of associative learning A simple, intuitive account – but very powerful • E.g., explains effects of contingency, learning curves, and cue
competition (blocking) Still often used as a “default” view of learning An alternative view “Many theorists maintain a belief in a learning mechanism in
which links between mental representations are formed
automatically… We conclude that learning is the consequence
of propositional reasoning processes that cooperate with the
unconscious processes involved in memory retrieval” An alternative view The link-based approach (e.g., Rescorla-Wagner) ILLV The propositional approach Last time I ate Vegemite I was ill. Perhaps Vegemite makes me ill. The propositional approach Last 2 times I ate Vegemite I was ill. Fairly sure Vegemite makes me ill. 0 0.5 1 0 5 10 15 St re ng th
o f b el ie f Trial The propositional approach • The “thinking about it” approach • Remember what went with what in the past, try and
work out most likely explanation
Blocking 0 1 2 3 4 Bread Dates Ju dg m en t r at in g • Previous learning about effectiveness of apples
blocks subsequent learning
about bread • Cues compete with each
other Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? Aitken, Larkin & Dickinson (2000, Expt 1) Propositions and blocking • Remember what went with what in the past, try and
work out most likely explanation
“If apple and bread together cause the outcome to occur with
the same probability as apple alone, this implies that bread is
not a cause of the outcome.” Phase 1 Phase 2 Test Apple → allergy Apple & Bread → allergy Bread? Carrot & Dates → allergy Dates? Blocking and trial order • According to the propositional account, order of the trials doesn’t matter • Should get blocking in the standard (forward) procedure, and in a backward
procedure Phase 1 Phase 2 Test A → outcome A + B → outcome B? C + D → outcome D? “If A and B together cause the outcome to occur with the same probability and
magnitude as A alone, this implies that B is not a cause of the outcome.” FO RW AR D Proposition Blocking BA CK W AR D Phase 1 Phase 2 Test A + B → outcome A → outcome B? C + D → outcome D? Blocking • In the Forward case, RW model says that blocking occurs because the outcome
is less surprising on “A+B” trials than “C+D” trials • In the backward case, the outcome is equally surprising on “A+B” trials and
“C+D” trials • So RW model predicts no blocking in backward condition Blocking and trial order Phase 1 Phase 2 Test A → outcome A + B → outcome B? C + D → outcome D?FO RW AR D Proposition Blocking BA CK W AR D Phase 1 Phase 2 Test A + B → outcome A → outcome B? C + D → outcome D? Blocking RW Blocking No blocking B D 0 2 4 6 8 Ju dg m en t r at in g ✱• We’ve already seen
blocking in the
“Forward” case – e.g., Aitken, Larkin &
Dickinson (2000), see
Lecture 1 • And it also occurs in the
“Backward” case! – e.g., Wasserman & Berglan (1998) • Consistent with propositional
account, but not with RW Blocking and trial order Phase 1 Phase 2 Test A → outcome A + B → outcome B? C + D → outcome D?FO RW AR D Proposition BA CK W AR D Phase 1 Phase 2 Test A + B → outcome A → outcome B? C + D → outcome D? Blocking Blocking RW Blocking No blocking “If A and B together cause the outcome to occur with the same
probability and magnitude as A alone, this implies that B is not
a cause of the outcome.” • In order to draw this inference, you need to know that B has
been paired with the outcome – Inferences based on memories of A→outcome and AB→outcome – B is paired with the outcome, but does not cause the outcome Blocking of memory Phase 1 Phase 2 Test A → outcome AB → outcome B? Causality Memory Propositional account Blocking of causal
judgement Good memory of
outcome Link-based account
(e.g. RW) Blocking of causal
judgement Blocking of memory
of outcome Phase 1 Phase 2 Test A → outcome AB → outcome B? Blocking of memory Phase 1 Phase 2 Test A → outcome AB → outcome B? A B outcome • RW:
Blocked cue does not form a
strong link with the outcome – People don’t encode that B was ever
paired with the outcome Blocking of memory Phase 1 Phase 2 Test A → outcome AB → outcome B? Blocking of memory Causality Memory Propositional account Blocking of causal
judgement Good memory of
outcome Link-based account
(e.g. RW) Blocking of causal
judgement Blocking of memory
of outcome Mitchell, Lovibond, Minard & Lavis (2006) Mr X ate Apple He suffered Xianethis Mitchell, Lovibond, Minard & Lavis (2006) Mr X ate Peach He suffered Daryosis Mitchell, Lovibond, Minard & Lavis (2006) Mr X ate Apple Breadand He suffered Xianethis Which illness followed Bread during training? Daryosis Xianethis Plonthema Eneuritis To what extent did the food cause this illness? 0 1 2 3 4 5 6 7 8 definitely
not causal possibly definitely
causal Blocking in ratings
of causality… … and in memory
of outcomes A → outcome AB → outcome B = Blocked CD → outcome D = Control Mitchell, Lovibond, Minard & Lavis (2006) Phase 1 Phase 2 Test A → outcome AB → outcome B? Blocking of memory Causality Memory Propositional account Blocking of causal
judgement Good memory of
outcome Link-based account
(e.g. RW) Blocking of causal
judgement Blocking of memory
of outcome Summary Propositions Links Backward
blocking Blocking of
memory of cue- outcome pairings Summary • It’s still not clear: More research needed! • Seems likely that both types of process operate • Simpler experiments encourage “thinking about it” • More complex experiments discourage thinking about it, so more
automatic link-formation mechanism dominates Wasserman & Berglan (1998) Backward blocking Supports propositional Mitchell et al (2006): Blocking of cue-outcome memory Supports link-based Summary • “Associative learning” may be due to more
than one process 51作业君版权所有
