Page 1 of 9
CONFIDENTIAL EXAM PAPER
This paper is not to be removed from the exam venue
Mathematics and Statistics
EXAMINATION
Semester 1 - Final, 2023
STAT3021-1 Stochastic Processes (Paper)
EXAM WRITING TIME: 2 hours
READING TIME: 10 minutes
EXAM CONDITIONS: Closed book: no reference materials/resources are
permitted. Main test
MATERIALS PERMITTED IN THE EXAM VENUE: (No electronic aids are
permitted e.g. laptops, phones) None
MATERIALS TO BE SUPPLIED TO STUDENTS: 1 x 12-page answer book
INSTRUCTIONS TO STUDENTS:
• There are FIVE questions; All questions are short answer questions;
No justification is required for Questions 1-3; Short justification is
required for Questions 4-5.
• Write your solutions in answer book, NOT in the exam paper.
• Notation and formula are given in pages 7-9.
Please tick the box to confirm that your examination paper is complete.
Room Number ________
Seat Number ________
Student Number |__|__|__|__|__|__|__|__|__|
ANONYMOUSLY MARKED
(Please do not write your name on this exam paper)
For Examiner Use Only
Q Mark
1
2
3
4
5
Total ________
Page 2 of 9
STAT3021-Stochastic Processes−Main
1. (10 marks. No justification is required.)
Consider a Markov chain{X
n
}
n≥0
having the following transition diagram:
12
5
3
467
1/2
1/2
1/2
1/4
1/4
(a) Find all closed classes.
(b) Find the mean recurrence time of state 3.
(c) Find the transition probabilityp
12
andp
76
.
(d) Find the period of state 7.
(e) Findf
45
.
Page 3 of 9
2. (12 marks. No justification is required.)
Two players, A and B, play the game of matching pennies: at each time, each player
has a penny and must secretly turn the penny to head or tail. The players then
reveal their choices simultaneously. If the pennies match (both heads or both tails),
Player A wins the penny. If the pennies do not match (one head and one tail),
Player B wins the penny. Suppose that two players start with 10 pennies each and
the game ends whenever Player A has 15 pennies or Player B has all 20 pennies.
(a) LetS
0
= 10 andS
n
,n≥1, be the number of pennies in which Player A has
before the (n+ 1)-th game. Explain why{S
n
}
n≥1
is a random walk with two
barriers and initial value 10 pennies. Find the two barriers.
(b) Find the probability that the game ends with Player A having 15 pennies.
(c) Find the average duration of the game.
(d) Suppose that Player B has infinitely many pennies, Player A still starts with
10 pennies and the game keeps going until Player A runs out of penny. Find
the probability that Player A is ruined and the average number of the game
when Player A is ruined.
Page 4 of 9
3. (8 marks. No justification is required)
Let{X
n
}
n≥1
be a branching process andqbe the extinction probability. Suppose
the offspringξ
10
has the pgf
F(s) =Es
ξ
10
= 1/3 +as
2
,
whereais unknown.
(a) Find the value ofa.
(b) AssumeX
0
= 2. Find the extinction probabilityq.
(c) FindE(X
n+1
|X
n
= 10).
(d) FindP(X
n+1
= 0|X
n
= 10).
Page 5 of 9
4. (18 marks. Short justification is required.)
On a work day, there are two independent buses A and B arriving at a local station.
The arrivals of buses form a Poisson process with rate 20 buses per hour. Among
the arrivals, 25% of the buses are Bus A and 75% are Bus B. Bus A takes 16 minutes
to get to work and Bus B takes 28 minutes to get to work. You always take the first
bus that arrives. Your co-worker always takes the first Bus A. You both are waiting
at the same station.
(a) Find the mean time between the arrivals of new buses.
(b) Find the related processes (N
1
(t) andN
2
(t), say) that describe the number of
Bus A and Bus B arrivals during (0,t].
(c) Find the probability that the first bus enters is Bus A.
(d) Find the probability that, in the first 15 mins, there is only one Bus B arrival..
(e) Find your expected arrival time to work.
(f) Find your co-worker’s expected arrival time to work.
Page 6 of 9
5. (12 marks. Short justification is required.)
Customers arriving to a cashier form a Poisson process with intensity 10 customers
per hour. A customer, upon arrival, will receive service immediately if the cashier
is free, otherwise she/he has to wait by joining a waiting line until serviced. The
service time is exponentially distributed with mean time of 5 mins. Let{L
t
}
t≥0
be
the number of customers (either waiting or being served) before the cashier, starting
at 9am. It is well known that{L
t
}
t≥0
is a continuous homogenous Markov chain
with state spaceS={0,1,2,...}and theQ-matrix:
Q=
−λ λ00...
μ−(λ+μ)λ0...
0μ−(λ+μ)λ ...
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
(a) Find the values ofλandμinQ.
(b) Find the probability that the cashier is idle in the long run.
(c) Find the average number of customers waiting (excluding being served) for
services in the long run.
(d) What is the probability that a customer must wait for more thant= 10 mins
before the service starts?
END OF QUESTIONS
Page 7 of 9
Notation and Formula
•Markov chain
–f
(1)
ij
=p
ij
=P(X
1
=j|X
0
=i);
–f
(n)
ij
=P(X
n
=j,X
n−1
6=j,...,X
1
6=j|X
0
=i),n≥2;
–f
ij
=
∑
∞
n=1
f
(n)
ij
.
•Random walk with barriers
S
n
=
∑
n
j=0
X
j
, whereX
0
=m >0 andX
j
,j≥1,are i.i.d r.vs with
P(X
1
= 1) =p, P(X
1
=−1) =q,0< p <1,p+q= 1.
{S
n
}
n≥1
satisfies the conditions thatS
n+1
= 0 ifS
n
= 0 andS
n+1
=kifS
n
=k
wherek > m. Let
N= min{n≥0 :S
n
= 0 orS
n
=k},
Ifp=q, givenX
0
=m, then
P(S
N
= 0) = (k−m)/k;P(S
N
=k) =m/k;EN=m(k−m).
Ifp6=q, givenX
0
=m, then
P(S
N
= 0) = 1−(1−θ
m
)/(1−θ
k
);
P(S
N
=k) = (1−θ
m
)/(1−θ
k
);
EN=
1
q−p
[
m−k(1−θ
m
)/(1−θ
k
)
]
,whereθ=q/p.
Page 8 of 9
•Branching process
X
1
=ξ
10
andX
n+1
=
∑
X
n
j=1
ξ
jn
I
(X
n
≥1)
, whereξ
jk
,k≥0,j≥1, are iid random
variables with distribution:
P(ξ
jk
=i) =f
i
, i= 0,1,2...,
∞
∑
i=0
f
i
= 1,0< f
0
<1,
Branching process is a Markov chain and
E(X
n
|X
0
=i) =iE(X
n
|X
0
= 1) =iμ
n
.
•Poisson process
The Poisson process{N
t
}
t≥0
withλ >0 is a process withN
0
= 0 and independent
increments, and for alls≥0 andt >0,
P(N
t+s
−N
s
=j) =
(λt)
j
e
−λt
j!
, j= 0,1,2,...
We can writeN
t
=N
1
(t) +N
2
(t), whereN
1
(t) andN
2
(t) are two independent
Poisson processes with ratesλ
1
andλ
2
so thatλ=λ
1
+λ
2
.
LetT
n
= inf{s >0 :N
s
=n}andE
n
=T
n
−T
n−1
(T
0
= 0). Then,
–E
j
,j≥1,are mutually independent withE
j
∼Exp(λ);
•Continuous-time MC{X
t
}
t≥0
Basic Assumption:
p
ii
(h) =P(X
h
=i|X
0
=i) = 1−λ
i
h+o(h),
p
ij
(h) =P(X
h
=j|X
0
=i) =q
ij
h+o(h),fori6=j .
Page 9 of 9
TheQ-matrix:
Q=
−λ
1
q
12
... q
1r
...
q
21
−λ
2
... q
2r
...
. . . .
. . . .
q
r1
q
r2
...−λ
r
...
. . . .
•The queueM/M/1
The arrival process{N
t
}is a Poisson process with rateλand the service time
Y∼Exp(μ). The traffic intensityρ=λ/μ.
LetL
t
denote the number of customers (either waiting or being served) andL=L
∞
.
Ifρ <1, the stationary dist and the limit dist of{L
t
,t≥0}are the same given by
π
k
= (1−ρ)ρ
k
, k≥0.
–EL=
∑
∞
j=0
jπ(j) =λ/(μ−λ);
–EL
∗
=
∑
∞
j=1
(j−1)π(j) =λ
2
/[μ(μ−λ)],whereL
∗
= max{L−1,0}= (L−
1)I(L≥1);
–LetWandVdenote the waiting time and the service time of a customer in
the queueM/M/1. Forx≥0,
P(W≤x) = 1−ρe
−μ(1−ρ)x
, P(W+V≤x) = 1−e
−μ(1−ρ)x
.
Also, we haveE(W) =ρ/[μ(1−ρ)] andE(W+V) = 1/[μ(1−ρ)].
THIS IS THE LAST PAGE
