PAPER CODE NO.
MATH 360
EXAMINER: Dr. Yi Zhang, TEL.NO. 44761
DEPARTMENT: Mathematical Sciences
MAY 2019 EXAMINATIONS
Applied Stochastic Models
Time allowed: Two and a half hours
INSTRUCTIONS TO CANDIDATES: Full marks will be given for complete
answers to FIVE questions. Only the best FIVE will be taken into account.
Paper Code MATH 360 Page 1 of 6 CONTINUED
1. (a) Consider a discrete-time Markov chain with the state space {1, 2, 3}
and the (one-step) transition probability matrix P given by
P =

 0 1 00 1
2
1
2
1
2
0 1
2

 .
Compute p
(n)
11 for each n ≥ 0. [13 marks]
(b) A system can be in one of the states 1, 2, 3 and 4. If the system is at state
j with j < 4, then, on the next step, it passes to state j + 1. From state 4,
independently on everything else, on the next step, the system passes either to 2
or to 3 with equal probabilities p˜42 = p˜43 =
1
2
.
(i) Model this system as a discrete-time Markov chain with the state space E
and the (one-step) transition matrix say P˜ = {p˜ij}i,j∈E, and compute P˜
2. (You
need explicitly state what E and P˜ are.) [3 marks]
(ii) Compute p˜
(16)
22 . [4 marks]
2. A machine produces two items every day. The probability that an item is
non-defective is p ∈ (0, 1) with p 6= 1
2
(all items are independent), and defective
items are thrown away instantly. The demand is one item per day, and any
demand that cannot be satisfied by the end of the day is lost, while any extra
item is stored. Let Xn be the number of items in storage just before the beginning
of the nth day.
(a) Model {Xn}n≥0 as a discrete-time Markov chain, that is, indicate its state
space and transition probability matrix. [4 marks]
(b) Decide for what values of p ∈ (0, 1) with p 6= 1
2
, {Xn}n≥0 is an ergodic
discrete-time Markov chain, and obtain the corresponding invariant distribution.
[8 marks]
(c) Decide for what values of p ∈ (0, 1), {Xn}n≥0 is a null recurrent discrete-
time Markov chain. [8 marks]
Paper Code MATH 360 Page 2 of 6 CONTINUED
3. (a) Consider a continuous-time Markov chain with the state space {1, 2}
and the Q-matrix given by
Q =
(
−α α
β −β
)
,
where α, β > 0. Compute p12(t) and p11(t) for each t ≥ 0. [12 marks]
(b) Consider {Xt}t≥0 and {Yt}t≥0 as two independent continuous-time Markov
chains on the common state space {1, 2} with the common Q-matrix
Q =
(
−α α
β −β
)
,
where α, β > 0. Decide with justifications whether {(Xt, Yt)}t≥0 is a continuous-
time Markov chain. If it is, indicate its state space and Q-matrix. [5 marks]
(c) Consider a continuous-time Markov chain with the state space {a, b, c, d}
and the Q-matrix given by
Q =


−2 1 0 1
1 −2 1 0
0 1 −2 1
1 0 1 −2


Compute pa,a(t) for all t ≥ 0. [3 marks]
Paper Code MATH 360 Page 3 of 6 CONTINUED
4. (a) Consider a birth-and-death process with birth rate λn > 0 for all
n = 0, 1, . . . , and death rate µn for all n ≥ 1. Consider the series
∞∑
n=1
(
1
λn
+
µn
λnλn−1
+ · · ·+
µnµn−1 . . . µ1
λnλn−1 . . . λ1λ0
)
.
State without proof the connection between the regularity of the birth-and-death
process with the convergence or divergence of the above series. [2 marks]
(b) For each of the following birth-and-death processes, decide with reasons
whether it is regular:
(i) λn = n+ 2, n ≥ 0; µn = n, n ≥ 1.
(ii) λn = n+ 1, n ≥ 0; µn = n, n ≥ 1.
(iii) λ0 = λ1 = 1, λn = n− 1, n ≥ 2; µn = n, n ≥ 1.
[9 marks]
(c) The following fact is known. For a discrete-time Markov chain on E =
{0, 1, . . . } with the transition probability given by
p0,1 = p0 = 1; pi,i+1 = pi ∈ (0, 1), pi,i−1 = qi = 1− pi, ∀ i ≥ 1,
it is recurrent if and only if
∞∑
k=1
qkqk−1 . . . q1
pkpk−1 . . . p1
=∞.
For each of the three birth-and-death processes in part (b), decide whether it is
recurrent. [9 marks]
Paper Code MATH 360 Page 4 of 6 CONTINUED
5. Let {Xt}t≥0 be a continuous-time Markov chain taking values in a finite
space {1, 2, 3, 4} with the Q-matrix given by
Q =


−3 1 1 1
0 −3 2 1
1 2 −4 1
0 0 1 −1

 .
Suppose P (X0 = 1) = 1. Calculate the expected time duration until the process
reaches state 2 for the first time, that is, the expected hitting time to state 2
given the initial state 1. [7 marks]
(b) Jobs arrive according to a Poisson process of rate λ > 0. They get
processed individually first-in-first-served, by a single processor, the processing
times being independent random variables, each with exponential distribution of
parameter ν > 0. After processing, a job either leaves the system with probability
p (0 < p < 1), or with probability 1 − p it is split into two separate jobs, which
are both sent to join the queue for processing again. Let Xt be the number of
jobs in the system at time t ≥ 0.
(i) Model {Xt}t≥0 as a continuous-time Markov chain, that is, indicate its state
space E and Q-matrix. Decide when, in terms of the values of p, ν, λ, this
continuous-time Markov chain is ergodic, and obtain limt→∞ P (Xt = j) for
each j ≥ 1.
[10 marks]
(ii) Compute the expected return time to state 0, given the initial state 0.
[3 marks]
Paper Code MATH 360 Page 5 of 6 CONTINUED
6. (a) Let {Zt}t≥0 be a (standard) Brownian motion. Check, with justifi-
cations, whether or not each of the following processes is a standard Brownian
motion.
(i) Ut = Z
2
t ;
(ii) Vt = Zt + Zt2 ;
(iii) Wt = Zt+2 − Z2.
[9 marks]
(b) Consider a population whose members cannot die, and each member is
independent of the others, and gives birth to descendants according to a Poisson
process with rate λ > 0. Let {Xt}t≥0 denote the population process. Assume that
the initial population is not empty. So we take the state space E = {1, 2, . . . }.
Show that the transition probability function of {Xt}t≥0 is given by
pij(t) =
(j − 1)!
(j − i)!(i− 1)!
e−iλt(1− e−λt)j−i
for each j ≥ i ≥ 1, and pij(t) = 0 if j < i. (Here you may use the fact that∑
j∈E pij(t) = 1 for each i ≥ 1 and t ≥ 0.) [7 marks]
(c) Let {Zt}t≥0 be a (standard) Brownian motion. Decide for what values of
θ ∈ R = (−∞,∞), Xt = e
θZt+
θ
2
t
2 defines a martingale with respect to the history
of the standard Brownian motion {Zt}. [4 marks]
7. Let {Wt}t≥0 be a standard Brownian motion.
(a) Solve the stochastic differential equation
dXt = Xtdt+ dWt
with some initial condition X0 = Z. [16 marks]
(b) Obtain the stochastic differential equation satisfied by {Xt}t≥0 with
Xt =
Wt
1 + t
.
[4 marks]
Paper Code MATH 360 Page 6 of 6 END

欢迎咨询51作业君