The University of Western Australia
School of Mathematics & Statistics
STAT3061 Random Processes & Applications
Assignment 1 (Semester 1, 2021)
• This assignment is assessed. Your work for this assignment must be submitted by 5:00pm
on Thursday 25th March, 2021.
• Submit your assignment in LMS, in PDF format: there will be an ‘Assignment 1’ tab un-
der ‘Assessments’. Submissions may be either handwritten or typed, but if handwritten,
make sure that it is neat and legible.
• Please ensure that you write your name and student number on each page of your work.
• Unless special considerations are granted, late submissions are penalised by 5% of the
total mark for each day late, up to 7 days. ‘Number of days late’ is a whole number
rounded up from the duration since the deadline has passed.
• To obtain full marks, you must demonstrate appropriate method and reasoning. Correct
answers may not necessarily earn full marks if they do not demonstrate an appropriate
level of understanding of the concepts and methods required.
• You may use R for matrix operations and for solving systems of equations. When referring
to such usage of R in your answers, please include the relevant R output and the source
code used.
• Plagiarism: the work you submit must be your sole effort (i.e. not copied from another
source). If you are found to have submitted work that is not your own, then you will be
subject to disciplinary action.
1. (3 + 3 + 4 + 10 = 20 marks) Consider a Markov chain with state space {1, 2, 3, 4}
and transition matrix P given below:
P =

0 1
3
2
3
0
1
2
0 0 1
2
1
4
0 0 3
4
0 1
2
1
2
0
 .
(a) Find the period of each state.
(b) For each state, decide whether it is recurrent or transient.
1
(c) Determine the stationary distribution for the Markov chain.
(d) For each of the following, demonstrate that the limit either does, or does not, exist. In
each case, find the limit if it exists. (Any argument that is solely based on numerical
evaluation of high powers of matrices using R will not be sufficient to confirm either
existence or non-existence of a limit, nor to evaluate an existing limit.)
i. lim
n→∞
Pn; ii. lim
n→∞
P2n; iii. lim
n→∞
P2n+1.
2. (3 + 3 + 4 = 10 marks) Recall the Bernoulli trial ‘word’ examples in Chapter 1.
Suppose we toss a fair coin repeatedly and independently until we reach the word HTHT.
(a) Draw an Engel diagram to reflect the relevant events that may occur, from the start
of the experiment up to the occurrence of the target word.
(b) Use your answer to (a) to formulate this experiment in Markov chain terms, by
choosing a state space of relevant stages and then writing the one-step transition
probability matrix.
(c) Write down the system of mean hitting equations for the target state, and solve
this system to determine the expected number of tosses needed to produce the word
HTHT.
3. (3 + 4 + 5 + 8 = 20 marks) Items arrive in a queue during time intervals (0, 1),
(1, 2), . . . . Let An denote the number of arrivals in the interval (n − 1, n), and assume
that (An : n ≥ 1) are mutually independent and identically distributed with pmf aj =
P (An = j) =
1
4
, j = 0, 1, 2, 3. The arriving items queue in a buffer with capacity 4:
if arrivals exceed the buffer capacity, then the excess is lost. A single server dispatches
one item per unit time (if any are waiting), with these dispatches occurring at times
n = 1, 2, . . . (by which is meant that the dispatch at time n occurs after the arrivals
An, and prior to the next arrivals An+1). Let Xn denote the buffer level at time n,
immediately before any dispatch. We know that (Xn) is a Markov chain, with state space
S = {0, 1, . . . , 4}. Denote by P = [pij]i,j the transition probability matrix for (Xn).
Suppose that the queue is empty initially, i.e. X0 = 0.
(a) Give the one-step transition probability matrix for (Xn).
(b) Find the mean occupation times (m0j(10) : j ∈ S) for (Xn).
(c) Find the expected time until the buffer reaches its capacity for the first time.
(d) Let Yn denote the number of items that go to waste from arrivals during the interval
(n − 1, n). Show that E(Yn+1 |X0 = 0) = 14p(n)0,3 + 34p(n)0,4 . [Hint: First evaluate
E(Yn+1 |Xn = j), for each j ∈ S, and then apply the Law of Total Expectation:
E(Yn+1 |X0 = 0) =
∑
j∈S E(Yn+1 |Xn = j, X0 = 0)P (Xn = j |X0 = 0).]
2

欢迎咨询51作业君