Module Code
MAT00018M
MMath and MSc Examinations 2019-20
Department:
Mathematics
Title of Exam:
Stochastic Processes
Time Allowed:
2 hours
Allocation of Marks:
The marking scheme shown on each question is indicative only.
Question: 1 2 3 4 Total
Marks: 28 23 16 33 100
Instructions for Candidates:
Answer all four questions.
Please write your answers in black or blue ink; pencil is acceptable for graphs and
diagrams. Do not use red ink.
You should simplify your answers as far as possible, but not resort to numerical
approximations.
No calculators may be used.
Materials Supplied:
Green booklet.
Do not write on this booklet before the exam begins.
Do not turn over this page until instructed to do so by an invigilator.
Page 1 (of 6)
MAT00018M
1 (of 4). (a) Give a definition of a Poisson process N = {N(t) : t ≥ 0} with rate λ. [4]
(b) Write down the formula for the transition probabilities
P (N(t) = j|N(s) = i)
for any t ≥ s ≥ 0 and j ≥ i ≥ 0. [4]
(c) Assuming that
P (N(1) = j) =
3j
j!
e−3, j = 0, 1, 2, 3, . . .
calculate, providing explanation for each step of your calculations,
E[N(1)].
[6]
Page 2 (of 6) continued on next page
continued from previous page MAT00018M
1 (of 4) cont.
(d) The photons emitted by an LED result from the recombination of electron-
hole pairs. Assume that the number of photons is modelled using a Poisson
process with rate of 10 photons per second.
(i) What is the probability that exactly 8 photons have been emitted during
the first 1 second? [2]
(ii) What is the probability that at most 1 photon has been emitted during the
first 1 second? [2]
(iii) What is the probability that at least 1 photon has been emitted during the
first 1 second? [2]
(iv) What is the probability that exactly 2 photons arrive during the first 1
second and exactly 3 photons arrive during the first 3 seconds? [2]
(v) It is known that exactly 4 photons have been emitted during the first 3
seconds. What is the probability that exactly 1 photon has been emitted
in the first 1 second? [2]
(vi) How many seconds would you expect to have to wait until the first photon
has been emitted? [2]
(vii) Alice was counting the emitted photons. After she has counted the first
1000 photons, she was replaced by her colleague Bob.
What is the probability that exactly 5 photons have been counted by Bob
during the first 2 seconds on his watch? [2]
State explicitly any properties of the Poisson process used in your calcula-
tions.
[Total: 28]
Page 3 (of 6) Turn over
MAT00018M
2 (of 4). (a) Give a definition of a Brownian Motion W = {W (t) : t ≥ 0}. [5]
(b) Assuming that W (t), for t > 0, has density
pt(x) =
1√
2pit
e−
x2
2t , x ∈ R,
calculate the probability
P (W (1) ≤ 0) .
[5]
(c) Assuming that W (t), for t > 0, has density
pt(x) =
1√
2pit
e−
x2
2t , x ∈ R,
show that for t ≥ 0 the following equalities hold:
E
[
e4W (1/2)
]
= e4,
E
[
(W (1))2
]
= 1,
E
[
(W (5))3e−(W (5))
2]
= 0.
State without proof any result you are using. [13]
[Total: 23]
Page 4 (of 6)
MAT00018M
3 (of 4). (a) Suppose that N = N(t), t ≥ 0 is a birth-death process with birth rates
λ0, λ1, λ2, · · · ≥ 0 and death rates µ1, µ2, · · · ≥ 0.
Write down the Master (or the forward Kolmogorov) equations for the tran-
sition probabilities
pij(t) = P
(
N(t) = j|N(0) = i), t ≥ 0,
where i, j are natural numbers such that j ≥ i ≥ 0.
[4]
(b) Assume that b, a > 0 are two fixed numbers. Consider a system consisting
of K telephone lines. We assume that K ≥ 3. If n lines are busy, where
n ≤ K, then with probability nbh, one of them will be freed within small
time interval of length h > 0. If n lines are busy, where n ≤ K − 1, then
with probability ah, a new call will arrive. The probability that within that
time interval two or more conversations will terminate or two or more calls
will arrive, is negligible.
Let N(t), for t ≥ 0, be the number of occupied telephone lines.
Assume that N = (N(t)), t ≥ 0, is a birth-death process.
Find explicit expressions for the birth and death rates of the process N .
[4]
(c) Explain why the stationary distribution for the birth-death process from the
previous part exists.
Find the stationary distribution for the birth-death process from the previous
part.
Hint: You can use a general formula for the stationary distribution of a birth-
death process without deriving it.
[8]
[Total: 16]
Page 5 (of 6) Turn over
MAT00018M
4 (of 4). Suppose that W = {W (t)}, t ≥ 0 is a Brownian Motion.
(a) Define a step process and then define the Itoˆ integral for such a process.
Let ξ = (ξ(t))t≥0 be a stochastic processes given by
ξ(t) =

1 if 0 ≤ t ≤ 3,
(W (2))2 if 3 < t ≤ 4,
W (4) if 4 < t ≤ 5,
0 if t > 5.
Explain why ξ is a step process. Evaluate the Itoˆ integral I(ξ) of ξ.
Note that the Itoˆ integral I(ξ) is also denoted by
∫ 5
0
ξ(s) dW (s).
Compute the mean value of this Itoˆ integral. [6]
(b) State (without a proof) the Itoˆ isometry. Compute the variance of this Itoˆ
integral I(ξ) from part (a).
[4]
(c) State the Itoˆ Lemma in the simple form.
Assume that T > 0 is a given number. Using the Itoˆ Lemma show that∫ T
0
tW (t) dW (t) =
1
2
T (W (T ))2 − 1
2
∫ T
0
(W (t))2 dt− 1
4
T 2.
[6]
Using the last result show that
E[
∫ T
0
tW (t) dW (t)] = 0.
[3]
(d) Using the Itoˆ Lemma or otherwise, find the solution to the following stochas-
tic differential equation
dX(t) = 3X(t)dt+ 4X(t) dW (t), (1)
with initial condition X(0) = 1.
[14]
[Total: 33]
Page 6 (of 6) End of examination.
SOLUTIONS: MAT00018M
1. (a) A Poisson process with rate λ > 0 is a stochastic process N = {N(t) : t ≥
0} taking values in S = {0, 1, . . . } such that:
a) N(0) = 0;
b) N has independent increments.
c) P(N(t+ h) = k + i|N(t) = k) =

λh+ o(h) if i = 1
o(h) if i > 1
1− λh+ o(h) if i = 0
0 if i < 0;
for any small h > 0. 4 Marks
(b) The transition probabilities for the Poisson process are given by
P (N(t) = j|N(s) = i) = P (N(t− s) = j − i)
=
(λ(t− s))j−i
(j − i)! e
−λ(t−s).
(2)
4 Marks
(c) In the following λ = 3.
E(N(1)) =
∞∑
k=0
xkpk =
∞∑
k=0
k
λk
k!
e−λ = e−λ
∞∑
k=1
λk−1λ
(k − 1)! = λe
−λ
∞∑
k=1
λk−1
(k − 1)!
= λe−λ
∞∑
n=0
λn
n!
(n = k − 1)
= λe−λeλ = λ = 3. 6 Marks
(d) We measure time in units equal to 1 second. The rate λ = 10. We put the
numerical value of λ at the last moment. We do not use calculator to make
calculations.
Note that for s = 0 and i = 0 the formula from part (b) takes the following
form
P (N(t) = j) =
(λ t)j
j!
e−λt. (3)
(i) We use formula (3) with t = 1, λ = 10 and j = 8, so that tλ = 10.
P(N(t) = j) =
(tλ)j
j!
e−tλ =
(tλ)8
8!
e−tλ =
108
8!
e−10
So
P(N(1) = 8) =
108
8!
e−10.
7
SOLUTIONS: MAT00018M
2 Marks
(ii) We calculate, with t = 1 and λ = 10,
P(N(t) ≤ 1) = P(N(t) = 0) + P(N(t) = 1)
We begin with find the first term. For this aim we again use formula (3)
with t = 1, λ = 10 (so that tλ = 10) but now with j = 0.
We have
P(N(t) = 0) =
(tλ)0
0!
e−tλ =
(tλ)0
0!
e−tλ =
1
1
e−10 = e−10
Next, we calculate the second term. By the same methods
P(N(t) = 1) =
(tλ)1
1!
e−tλ =
(tλ)1
1!
e−tλ = λe−λ = 10 e−10
Hence, we infer that
P(N(1) ≤ 1) = 10 e−10 + e−10 = 11 e−10 2 Marks
(iii) We calculate, using properties of the probability measure and the result
from the last part to get
P(N(t) ≥ 1) = 1− P(N(t) < 1) = 1− P(N(t) = 0)
= 1− e−tλ = 1− e−10,
where the last but one equality follows from the previous part. 2 Marks
(iv) Put s = 1 and i = 2, t = 3 and j = 3. Then we will have. The most
important step is write the formula for the desired probability using the
increments:
P(N(s) = i, N(t) = j) = P(N(s) = i, N(t)−N(s) = j − i)
Because the r.v.’s N(t)−N(s) and N(s) are independent
= P(N(s) = i)P(N(t)−N(s) = j − i)
by applying formula (3) twice
=
(sλ)i
i!
e−sλ
(
(t− s)λ)j−i
(j − i)! e
−(t−s)λ
=
(λ)2
2!
e−λ
(
2× λ)1
1!
e−2×λ = λ3e−3λ
= 103e−30
8
SOLUTIONS: MAT00018M
So
P(N(1) = 2 and N(3) = 3) = 103e−30 2 Marks
(v) By the definition of the conditional expectation we have, with s = 1,
t = 3, i = 1 and j = 4, we have
P(N(1) = 1|N(3) = 4) = P(N(1) = 1, N(3) = 4)
P(N(3) = 4)
=
P(N(1) = 1, N(3)−N(1) = 4− 1)
P(N(3) = 4)
by the independe of r.v.’s N(3)−N(1) and N(1)
=
P(N(1) = 1)P(N(3)−N(1) = 4− 1)
P(N(3) = 4)
=
(1×λ)1
1!
e−1×λ × [(3−1)×λ]4−1
(4−1)! e
−(3−1)×λ
(3×λ)4
4!
e−3×λ
=
λe−λ × [2λ]3
3!
e−2λ
(3×λ)4
4!
e−3×λ
since the exponentials cancel =
λ × [2λ]3
3!
(3×λ)4
4!
=
23 × 4!× λ4
34 × 3!× λ4
since the λ4 cancel =
25
34
=
32
81 2 Marks
(vi) We define
τ1 = the smallest time t : N(t) = 1
This is a stopping time and we know from Lectures, Theorem 3.5, that
τ1 has an exponential distribution with parameter λ = 10. Hence
E(τ1) =
1
λ
=
1
10
.
2 Marks
(vii) We put
M(t) = N(t+ τ1)−N(τ1), t ≥ 0.
By Theorem 3.4, M = (M(t)), t ≥ 0 is also a Poisson process with the
same rate λ = 10 as N . Therefore, by formula (3) with t = 2, λ = 10
and j = 5 we get
9
SOLUTIONS: MAT00018M
P(M(t) = 5) =
(tλ)5
5!
e−tλ =
(20)5
5!
e−20.
2 Marks
Total: 28 Marks
2. (a) A Brownian Motion W = {W (t) : t ≥ 0} is a stochastic process such that
(i) W (0) = 0 a.s.;
(ii) For any s ≥ 0, t > 0,W (t+ s)−W (s) ∼ N(0, t);
(iii) W has independent increments;
(iv) The sample paths of W are continuous a.s.. 5 Marks
(b) We have
P (W (1) ≤ 0) = P (W (1) ∈ (−∞, 0]) =
∫ 0
−∞
p1(x)dx
=
∫ 0
−∞
1√
2pi
exp
(
−x
2
2
)
dx
=
1
2
[ ∫ 0
−∞
1√
2pi
exp
(
−x
2
2
dx
)
+
∫ ∞
0
1√
2pi
exp
(
−x
2
2
dx
)]
=
1
2
∫ ∞
−∞
1√
2pi
exp
(
−x
2
2
)
dx =
1
2
.
We used the fact that the function f : x 7→ 1√
2pit
exp
(
−x2
2t
)
is even so that
∫ 0
−∞
f(x) dx =
∫ ∞
0
f(x) dx.
The last equality was stated in the Lectures. 5 Marks
(c) We start with the most difficult part. Since the r.v. W (t), has density
pt(x) =
1√
2pit
e−
x2
2t , x ∈ R,
we get, by the change of measure Theorem,
10
SOLUTIONS: MAT00018M
E
[
e4W (t)
]
=
∫ ∞
−∞
e4xpt(x) dx
=
∫ ∞
−∞
e4x
1√
2pit
e−
x2
2t dx =
1√
2pit
∫ ∞
−∞
e4x−
x2
2t dx
Now we have, for any λ ∈ R,
1√
2pit
∫ ∞
−∞
eλx−
x2
2t dx = . . .
λx− x
2
2t
= − 1
2t
[
x2 − 2tλx+ (tλ)2 − (tλ)2
]
= − 1
2t
[(
x− λt)2 − (tλ)2]
= −(x− tλ)
2
2t
+
tλ2
2
. . . = e
tλ2
2
1√
2pit
∫ +∞
−∞
e−
(x−tλ)2
2t dx, y =
x− tλ√
t
, dx =
√
tdy
= e
tλ2
2
(
1√
2pi
∫ +∞
−∞
e−
x2
2 dx
)
= e
tλ2
2 .
Putting λ = 4 and t = 1/2 we deduce that
E
[
e4W (1/2)
]
= e
1/2×42
2 = e4.
A solution to the last case is based on the same principle that an integral∫∞
−∞ f(x) dx is equal to 0 when f is an odd function.
As above, by the change of measure Theorem,
E
[
(W (t))3e−(W (t))
2]
=
∫ ∞
−∞
x3e−x
2
pt(x) dx
=
∫ ∞
−∞
x3e−x
2 1√
2pit
e−
x2
2t dx = 0
because the function
f(x) = x3e−x
2
e−
x2
2t
is odd.
The second case is the most difficult.
Again by the change of measure Theorem with g(x) = x2 to conclude that
E
[
W (1)2
]
= E
[
g(W (1))
]
=
∫ +∞
−∞
g(x)p1(x) dx =
∫ +∞
−∞
x2p1(x) dx
=
∫ +∞
−∞
x2√
2pi
e−
x2
2 dx =
1√
2pi
∫ +∞
−∞
x2e−
x2
2 dx.
11
SOLUTIONS: MAT00018M
By the integration by parts formula we have
we have∫ +∞
−∞
x2e−
x2
2 dx = lim
n→∞
∫ n
−n
x2e−
x2
2 dx = lim
n→∞
∫ n
−n
(−x)(−xe−x
2
2 )dx
= lim
n→∞
∫ n
−n
(−x)(e−x
2
2 )′dx = lim
n→∞
[
− xe−x
2
2 |n−n −
∫ n
−n
−e−x
2
2 dx
]
= lim
n→∞
2n
e
n2
2
+
∫ +∞
−∞
e−
x2
2 dx =
∫ +∞
−∞
e−
x2
2 dx, (4)
where lim
n→∞
2n
e
n2
2
= 0 by l’Hospital rule.
Therefore we infer that
E
[
W (1)2
]
=
1√
2pi
∫ +∞
−∞
e−
x2
2 dx = 1.
13 Marks Total: 23 Marks
3. (a) Recall that birth-death process with birth rates λ0, λ1, . . . and death rates
µ1, µ2, . . . , where λi ≥ 0 and µi ≥ 0, is a Markov process N = {N(t) : t ≥
0} with state space N such that for any t ≥ 0 and any small h ≥ 0,
P(N(t+ h) = n+m|N(t) = n) =

λnh+ o(h) if m = 1
µnh+ o(h) if m = −1
o(h) if m > 1
1− (λn + µn)h+ o(h) if m = 0
0 if |m| ≥ 2.
The master equation expresses the rate of change in the probability of being
in a state n as the difference between the rate at which the system enters the
state and the rate at which it leaves the state. In the case of a birth-death
process this gives
d
dt
pij(t) = λj−1pi,j−1(t) + µj+1pi,j+1(t)− (λj + µj)pij(t). (5)
where we use the convention from the lectures that λ−1 = µ0 = p−1 = 0. 4 Marks
12
SOLUTIONS: MAT00018M
(b) In this problem, the births are represented by a new call arriving and the
deaths are represented by a line being freed. Thus the birth rates are
λn =
{
a, for n = 0, 1, 2, · · · , K
0, for n = K,K + 1, · · · .
and the death rates are
µn =
{
µb, for n = 0, 1, 2, · · · , K
0, for n = K,K + 1, · · · .
4 Marks
(c) The stationary distribution for this birth-death process exists because
∞∑
n=1
λ0λ1 · · ·λn−1
µ1µ2 · · ·µn =
K∑
n=1
λ0λ1 · · ·λn−1
µ1µ2 · · ·µn <∞ (6)
In our case,
λn−1
µn
=
a
nb
Hence,
K∑
n=1
λ0λ1 · · ·λn−1
µ1µ2 · · ·µn =
K∑
n=1
a
b
· · · a
nb
=
K∑
n=1
(
a
b
)n
1
n!
The stationary distribution exists and can be calculated as follows.
pi0 =
(
1 +
K∑
n=1
(
a
b
)n
1
n!
)−1
pin =
λ0λ1 · · ·λn−1
µ1µ2 · · ·µn pi0 =
1
n!
(a
b
)n(
1 +
K∑
n=1
(
a
b
)n
1
n!
)−1
, 1 ≤ n ≤ K.
(7)
8 Marks Total: 16 Marks
4. (a) Definition Let (Ω,F ,P) be a probability space and let W = W (t)t≥0 be
a Brownian Motion w.r.t a filtration (Ft)t≥0. A step process is a process
(ξ(t))t≥0 satisfying the following conditions
(i) (ξ(t))t≥0 is adapted to (Ft)t≥0,
13
SOLUTIONS: MAT00018M
(ii) E|ξ(t)|2 <∞ for all t ≥ 0,
(iii) there exists a finite sequence (t0, . . . , tN) such that 0 = t0 < t1 < · · · <
tN = T and
ξ(t) =
{
ξ(ti), if t ∈ [ti, ti+1) for i = 0, . . . , N − 1
0, if t ≥ tN .
For such a step process ξ = (ξ(t))t≥0 we define its Itoˆ integral by
I(ξ) =
N−1∑
i=0
ξ(ti)(W (ti+1)−W (ti)). (8)
The stochastic process ξ is a simple process as well, with N = 3 and t0 =
0 < t1 = 3 < t2 = 4 < t3 = 5. Then by formula (8) we have
Y = I(ξ) =
N−1∑
i=0
ξ(ti)(W (ti+1)−W (ti)) =
3−1∑
i=0
ξ(ti)(W (ti+1)−W (ti))
= 1(W (3)−W (0)) + (W (2))2(W (4)−W (3)) +W (4)(W (5)−W (4))
By the properties of the Itoˆ integral
E
[
I(ξ)
]
= 0.
Alternatively, one can calculate
E
[
I(ξ)
]
= E
[
1(W (3)−W (0)) + (W (2))2(W (4)−W (3))
+ W (4)(W (5)−W (4))
]
= · · · = 0.
Remarks. Bookwork and Unseen 6 Marks
(b) Theorem [Itoˆ isometry for step processes)] For a step process ξ = (ξ(t))t≥0
we have
E
(
[I(ξ)]2
)
= E([
∫ ∞
0
ξ(t)dW (t)]2) = E(
∫ ∞
0
|ξ(s)|2dt). (9)
We use the Itoˆ isometry to calculate E(Y 2).
E(Y 2) = E(|I(ξ)|2) =
∫ ∞
0
E|ξ(t)|2 dt
=
∫ 5
0
E|ξ(t)|2 dt+
∫ ∞
5
E|ξ(t)|2 dt
=
∫ 5
0
E|ξ(t)|2 dt.
14
SOLUTIONS: MAT00018M
Therefore,
E(Y 2) =
∫ 5
0
E|ξ(t)|2 dt =
3−1∑
i=0
E(|ξti+|2)(ti+1 − ti)
= E(1)2(3− 0) + E((W (2))2)2(4− 3) + E(W (4))2(5− 4)
= 1 · 3 + 3 · 4 · 1 + 4 · 1 = 19.
Remarks. Bookwork and Unseen 4 Marks
(c) Theorem (The simple version of the) Itoˆ Lemma
Assume that f(t, x) is of C1,2 class. Define η(t) = f(t,W (t)), t ≥ 0. Then
η(t) is an Itoˆ process and for t ≥ 0, one has
η(t) = f(0, 0) +
∫ t
0
[∂f
∂s
(s,W (s)) +
1
2
∂2f
∂x2
(s,W (s))
]
ds
+
∫ t
0
∂f
∂x
(s,W (s)) dW (s).
We apply the above result for a function
f(t, x) = t
1
2
x2
which obviously satisfies the above assumptions. Note that f only depends
on the x variable and thus
∂f
∂t
=
1
2
x2,
∂f
∂x
= tx,
1
2
∂2f
∂x2
= t, f(0, 0) = 0.
Thus, for every T ≥ 0,
1
2
T (W (T ))2 = f(0, 0) +
∫ T
0
[∂f
∂t
(t,W (t)) +
1
2
∂2f
∂x2
(t,W (t))
]
dt
+
∫ T
0
∂f
∂x
(t,W (t)) dW (t)
= 0 +
∫ T
0
[1
2
(W (t))2 +
1
2
t
]
dt+
∫ T
0
tW (t) dW (t)
=
1
2
∫ T
0
(W (t))2 dt+
1
4
T 2 +
∫ T
0
tW (t) dW (t)
Rearranging, we get∫ T
0
tW (t) dW (t) =
1
2
T (W (T ))2 − 1
2
∫ T
0
(W (t))2 dt− 1
4
T 2.
15
SOLUTIONS: MAT00018M
6 Marks
Hence, since E
[
(W (t))2
]
= t so that E
[ ∫ T
0
(W (t))2 dt
]
= T
2
2
we deduce that
E
[∫ T
0
tW (t) dW (t)
]
= 0.
3 Marks
(d) Method based on use of Itoˆ formula. We look for a solution of the form
X(t) = f
(
t,W (t)
)
, t ≥ 0
for an appropriate function f(t, x) (to be found).
The simplified version of Itoˆ’s formula then gives, because ∂f
∂t
(t, x) = 0 and
f(t, x) = f(x),
dX(t) =
(
∂f
∂t
(t,W (t)) +
1
2
∂2f
∂x2
(t,W (t))
)
dt+
∂f
∂x
(t,W (t))dW (t)
Comparing the above with equation to be solved, i.e.
dX(t) = 3X(t)dt+ 4X(t) dW (t)
we deduce that f should satisfy, since X(t) = f
(
W (t)
)
,
∂f
∂x
(t,W (t)) = 4f
(
t,W (t)
)
∂f
∂t
(t,W (t)) +
1
2
∂2f
∂x2
(
W (t)
)
= 3f
(
W (t)
)
We can replace W (t) above by x to deduce
∂f
∂x
(x) = 4f(t, x)
∂f
∂t
(t, x) +
∂2f
∂x2
(x) = 3f(t, x)
Denoting y(x) = f(t, x), for fixed t, the first of these two equations becomes
dy
dx
= 4y(x)
Solving it we get
16
SOLUTIONS: MAT00018M
y(x) = ze4x
where, obviously, the constant z depends on t. So we get
f(t, x) = z(t)e4x
We need to find z(t). For this purpose we use the first of the above two
equations. We get
z′(t)e4x +
1
2
× 42z(t)e4x = 3z(t)e4x
Dividing by e4x we get
z′(t) = −5z(t)
This is a linear homogeneous equation whose solution is
z(t) = z0e
−5t
Therefore,
f(t, x) = z0e
−5te4x
and so
X(t) = z0e
−5t+4W (t)
Since X(0) = 1, we infer that
X(t) = e−5t+4W (t) 14 Marks Total: 33 Marks
17

欢迎咨询51作业君