MATH 452/STAT 552
Assignment 4
Due: November 17th, 2020 at the beginning of class.
Submission: Upload your solutions of Exercises 4.1–4.5 to Crowdmark as one single
file. (You do not have to hand in solutions to the Supplementary Exercises.)
Exercise 4.1. Consider a continuous-time Markov chain on E = {1, 2, 3, 4} such
that (1) the holding time of any i ∈ E is exponential with parameter i, and (2) at the
end of the holding time, the Markov chain jumps to one of the other states according
to the discrete uniform distribution. Find the Q-matrix of this Markov chain.
Exercise 4.2. Show that every birth and death process is reversible, whenever the
stationary distribution exists [1, pp.396–397]. This property is explained in the proof
of [1, Proposition 6.5 on p.403], but give a direct proof by checking piiqij = pijqji for
all i 6= j.
Exercise 4.3. Consider a continuous-time Markov chain with transition probabilities
Pij(t), and write Q for the Q-matrix. It was discussed in class that a stationary
distribution pi can be found by solving piQ = 0, and that the identity piP (t) = pi is
implied by the property that stationary probabilities arise as limiting probabilities.
Now, by using Kolmogorov’s backward equation and the identity P (0) = Id (the
identity matrix), show that piQ = 0 if and only if piP (t) = pi for all t ≥ 0.
The next two exercises show that in some cases, we can circumvent analytic meth-
ods for solving probabilities and expectations of continuous-time Markov chains by
turning to probabilistic representations. Recall that Exercise 2.4 has the same flavor.
Exercise 4.4. Let (N(t); t ≥ 0) be a Poisson process with rate λ and ξ1, ξ2, · · · ,
be i.i.d. {−1,+1}-valued random variables with P(ξ1 = +1) = p ∈ (0, 1). Assume
that the process and the random variables are independent. Define a continuous-time
Markov chain X by X(t) =
∑N(t)
n=1 ξn, and set M(t) = E[X(t)].
(1) FindM(t) by computing the expectation as an expectation of a compound sum.
(2) Extend the corresponding argument discussed in class for the linear growth
model with immigration [1, Example 6.4 on pp.378–380] to show that M(t)
satisfies a first-order differential equation taking the form
dM
dt
= aM + b, M(0) = 0, (4.1)
1
for explicit constants a, b. Then verify your solution in (1) by checking the two
equalities in (4.1).
Exercise 4.5. In this exercise, we consider a Poisson-process-like probabilistic repre-
sentation of the transition probabilities Pij(t) of the pure birth process [1, pp.393–394].
By relabelling indices if necessary, we can restrict attention to Pij(t) with i = 0. In
this case, we only need to work with two variables, j and t. A closely related question
can be found in [1, Exercise 6.11].
Given birth rates β0, β1, β2, · · · ∈ (0,∞), let e0, e1, e2, · · · be independent expo-
nential variables such that E[en] = 1/βn. We think of en as the arrival time of the
(n + 1)-th individual, given that there are already n individuals in the population.
Since P0j(t) is the probability that there are j individuals in the population at time
t, P0j(t) is equal to
Qj(t) = P
(
j−1∑
n=0
en ≤ t <
j∑
n=0
en
)
, 0 ≤ j <∞, (4.2)
with the convention that
∑−1
n=0 ≡ 0. The following questions study the explicit forms
of P0j(t) in terms of Qj(t).
(1) Let X and Y be independent, nonnegative random variables. By using P(X ≤
t < X + Y ) = E[1[X,X+Y )(t)], show that∫ ∞
0
e−µtP(X ≤ t < X + Y )dt = 1
µ
E[e−µX − e−µ(X+Y )], ∀ µ ∈ (0,∞).
(2) Use (1) to show that the Laplace transform of Qj for any j ≥ 0 is given by∫ ∞
0
e−µtQj(t)dt =
j−1∏
n=0
βn
µ+ βn
· 1
µ+ βj
, ∀ µ ∈ (0,∞), (4.3)
with the convention that
∏−1
n=0 ≡ 1.
Supplementary Exercises: 2, 7, 11, 46, 47 in [1, Chapter 6].
References
[1] Ross, S. (2019). Introduction to Probability Models. 12th edition. Academic Press.
2

欢迎咨询51作业君