GSOE9210 Engineering Decisions
Engineering Decisions
Bayes decisions
1 Decisions with likelihoods Modelling probabilistic actions
2 Bayes decisions
Bayes indifference classes Bayes, ignorance, and mixing
Victor Jauregui
Engineering Decisions
Bayes decisions
1 Decisions with likelihoods Modelling probabilistic actions
2 Bayes decisions
Bayes indifference classes Bayes, ignorance, and mixing
Victor Jauregui
Engineering Decisions
Decisions with likelihoods
Decisions with likelihoods
Decision problem classes
Decision problems can be classified based on an agent’s epistemic state:
Decisions under certainty: the agent knows the unique actual state Decisions under uncertainty:
Decisions under ignorance (full uncertainty): the agent believes multiple states/outcomes are possible; likelihoods unknown
Decisions under risk: the agent believes multiple states/outcomes are possible; likelihood information available
Engineering Decisions
Victor Jauregui
Decisions with likelihoods
River example
X A
Example (River logistics)
BC
Alice’s warehouse is located at X on a river that flows down-stream from C to A. She delivers goods to a client at C via motor boats. On some days a (free) goods ferry (f) travels up the river, stopping at A then B and C, but not at X.
The fuel required (litres) to reach C from each starting point: AXBC
ToCfrom: 4 3 2 0 Alice wants to minimise fuel consumption (in litres).
Victor Jauregui Engineering Decisions
Decisions with likelihoods
Likelihood information
Example (Ferry likelihood)
Suppose that Alice has to deliver one package to C every day. Her records show that out of the last 100 days, the ferry was operating on 75.
Additional information (Alice’s records) can be used to estimate likelihood of ferry being operational on any given day
Only general information
How might this affect Alice’s decision?
Victor Jauregui
Engineering Decisions
River example
Decisions with likelihoods
X A
f f
A 4 0 B C B31 C 1 1
Alice considers three possible ways to get to C (from starting point X):
A via A, by floating down the river
B via B, by travelling up-stream to B
C by travelling all the way to C
Outcomes are measured in litres left in a four-litre tank. Exercise
Let w : Ω → R denote fuel consumption in litres. What transformation f : R → R is responsible for the values v : Ω → R in the decision table?
Victor Jauregui Engineering Decisions
Decisions with likelihoods
Single decision; multiple trials
Long term fuel savings:
f f Avg min A 4 0 3 0
B 3 1 2.5 1
Short-term outcome horizon (one/a few days):
ff ff ff ff
A 8 4 4 0 B 6 4 4 2
ff ff ff ff
A 4 2 2 0 B 3 2 2 1
Sensible to use Maximin given likelihood of least favourable state (ff)?
Victor Jauregui
Engineering Decisions
Decisions with likelihoods
Single decision; multiple trials
Simplifying assumptions:
Assume long sequence of days and maximum likelihood probability
Infer probability that ferry operates on any given day: p = 75 = 3 100 4
f f E min A 4 0 3 0
B 3 1 2.5 1
Assume long-term outcome horizon
Victor Jauregui
Engineering Decisions
Decisions with likelihoods
Likelihood and decisions
Alice’s long-run total/average value is greater via A than B
Summary:
In this situation there are multiple trials (days) of some random process (ferry operation)
Different states may occur in each trial (day): ferry (f) or no ferry (f) Information available about ‘likelihood’ of occurrence of states:
75% ferry to 25% no ferry
Maximin assumes worst case for each action even when the worst case (no ferry) is unlikely
Would like a decision rule which takes likelihood information into account
Victor Jauregui
Engineering Decisions
Decisions with likelihoods Modelling probabilistic actions
Probabilistic lotteries
Definition (Probabilistic lottery)
A probabilistic lottery over a finite set of outcomes, or prizes, Ω, is a pair l = (Ω,P), where P : Ω → R is a probability function. The lottery l is written:
l = [p1 : c1|p2 : c2|...|pn : cn] where for each si ∈ S ⊆ P(Ω), pi = P(si) = P(ci).
Example (To C via A)
Alice’s decision to travel via A corresponds to:
34 : f 4
14 : f
lA = [34 : 4|14 : 0]
where outcomes have been replaced by their values.
Victor Jauregui
Decisions with likelihoods
Value of a lottery
Definition (Value of a lottery)
Engineering Decisions
Modelling probabilistic actions
The value of a probabilistic lottery (Ω, P, v) is the expected value over its outcomes:
For strategy A:
Vv(l) = E(v) = X P(ω)v(ω) ω∈Ω
V (lA) = 34 (4) + 14 (0) = 3 + 0 = 3
Note: not value of any outcome of strategy A: 4, 0
Frequency interpretation: V (lA) is the average value of A over many days
Victor Jauregui
Engineering Decisions
Bayes decisions
1 Decisions with likelihoods Modelling probabilistic actions
2 Bayes decisions
Bayes indifference classes Bayes, ignorance, and mixing
Victor Jauregui
Engineering Decisions
Bayes decisions
Bayes decisions
Bayes decisions
Under risk, each strategy in a decision problem corresponds to a probabilistic lottery.
Definition (Bayes value)
Given a probability distribution over states, the Bayes value, VB, of a strategy is the expected value of its outcomes.
Definition (Bayes strategy)
A Bayes strategy is a strategy with maximal Bayes value. Definition (Bayes decision rule)
The Bayes decision rule is the rule which selects all the Bayes strategies.
Engineering Decisions
Victor Jauregui
Bayes decisions Bayes strategies: River problem
Assume probability of ferry operating on an arbitrary day is P(f) = p:
p 1−p
f f VB A 4 0 4p
B 3 1 2p+1
Bayes values for each strategy plotted for all values of p ∈ [0, 1].
Exercise
For what values of p will the Bayes decision rule prefer A to B?
VB 5
4 A 3
B
p
Victor Jauregui
Engineering Decisions
Bayes decisions
Bayes indifference classes
Indifference curves: Maximin
For the pure actions below:
s1 s2 A 2 3
B 4 0 C 3 3 D 5 2 E 3 5
s2 5E
4 3AC3
2D2 I (A)
1 Consider curves of all points 0
representing strategies with same Maximin value; i.e., Maximin indifference curves.
0 1 2 3 4 5 s1
Victor Jauregui
Engineering Decisions
Bayes decisions Bayes indifference classes Indifference curves: Bayes
What do Bayes indifference curves look like?
p 1−p
ff VB A40 4p B31 2p+1
a v1 v2 pv1 + (1 − p)v2 Indifference curves:
f
VB(a)=pv1 +(1−p)v2 =u
In gradient-intercept form, v2 = u − p v1, where m = − p ; e.g.,
1−p 1−p 1−p Because v2 ∝ u; i.e., ‘higher’ lines receive greater Bayes values
forp=34,m=−43/14 =−31
B 0A
f
p = 34
p = 14
Victor Jauregui
Engineering Decisions
Bayes decisions
Bayes indifference classes
Indifference curves: Bayes
In general, for two actions:
f
Bb1 b2
∆y = |a2 − b2|
=m−1
where m is the gradient of line AB.
Forexample: ifAis(1,3)andBis (2, 1) then:
p= 3−1 (2−1)+(3−1)
=2=2 1+2 3
A
∆x=|a1−b1|
∆y
B
∆x
p 1−p
s1 s2 p= ∆y Aaa ∆x+∆y
12m
f
p=− ∆y
∆x+∆y
Victor Jauregui
Engineering Decisions
Bayes decisions Bayes indifference classes Indifference classes and Bayes decisions
Exercises
Prove expression for p in terms of gradient m
For river problem, what is slope of line joining the two actions? For what probability are the two actions of equal Bayes value? What is the Bayes value associated with this line?
Repeat the above exercises for regret
Victor Jauregui
Engineering Decisions
Bayes decisions
Bayes indifference classes
Bayes strategies
For the pure actions below with
P(s1) = p: s2
5C
3A
1B
s1s2 VB A233−p4
B 5 1 1+4p C 3 5 5−2p
p = 32
SlopeofBC:m=5−1 =−2.
3−5 ∴p=2=2.
2+1 3
s1
Note: p ∝ −m.
Victor Jauregui
Engineering Decisions
Bayes decisions Bayes indifference classes Bayes strategies: probability plots
For the pure actions below with P(s1) = p:
s1s2 VB A233−p
B 5 1 1+4p C 3 5 5−2p
For p = 23, the value of the Bayes action(s) is least.
Definition
VB
5C 4
B
A
p
The least favourable probability distribution on the states/outcomes is the probability distribution for which Bayes strategies have minimal values.
Victor Jauregui
Engineering Decisions
Bayes decisions
Bayes, ignorance, and mixing
Bayes solutions
For the pure actions below with P(s1) = p:
s1s2 VB A 1 5 5−4p
B 4 1 1+3p C344−p
SlopeofBC:m=4−1 =−3.
s2 5
A
C
M
p = 13
p = 12
p = 34
∴ p = 43 .
Slope of AC: m = −1. ∴p=3.
3−4
B
s1
Victor Jauregui
Engineering Decisions
Bayes strategies s2
VB
s01113 1043241
p
Bayes summary
Theorem
Results about Bayes decision rule:
B
Bayes decisions
Bayes, ignorance, and mixing
A 4C4CB
A
aM3M
The Maximin action is a Bayes action when p = 34 Mixed strategy a ∼ 0.5A0.3B0.2C is not Bayes
Victor Jauregui Engineering Decisions
Bayes decisions Bayes, ignorance, and mixing
Mixing can improve upon the Maximin value of pure strategies, but it does not improve upon the Bayes value of pure strategies
Bayes strategies are invariant/preserved under regret; i.e., the same strategy is chosen under regret as otherwise
Exercise
Prove the theorems above.
a
p = 34
p = 31
p = 12
Victor Jauregui
Engineering Decisions
Bayes decisions Bayes, ignorance, and mixing Bayes, Maximin, and admissibility
s2 s1 s2 4
A 0 4 3
B 3 1 2
C 2 3 1
D 1 2 Exercises
s1
A
C
DM B
Which mixed strategies above are admissible?
Are Maximin mixed strategies always admissible?
Are Bayes mixed strategies always admissible?
Are Maximin mixed strategies always Bayes for some p? Are admissible mixed strategies Bayes for some p?
Victor Jauregui
Engineering Decisions
Bayes decisions
Bayes, ignorance, and mixing
Bayes summary
Decision problems with partial (likelihood) information
Bayes decision rule logical when likelihood information available Bayes values, Bayes strategies, Bayes decision rule
Graphical (visual) representation of Bayes strategies/values Bayes indifference curves
Unresolved issues: short outcome horizon
Victor Jauregui
Engineering Decisions
