CONFIDENTIAL THE UNIVERSITY OF SYDNEY FACULTY OF SCIENCE SCHOOL OF MATHEMATICS AND STATISTICS STAT3021 Stochastic Processes —- Main June, 2021 (Semester 1) 

INSTRUCTIONS Exam Duration: 2 hrs + 10 mins reading time; 15 mins will be allocated to upload handwritten elements into Canvas (add link to assignment dropbox). Exam Conditions: Open book. Detailed Instructions: 1. This exam has 4 questions, together with one student declaration statement. You should attempt all questions and follow the instructions for each question carefully. 2. Please review the Student Charter and respond. Responding YES to the dec- laration means that you undertake the exam in the prescribed exam conditions with no assistance from a third party or the use of prohibited resources. 3. This exam must be taken on a computer or laptop with satisfactory internet connectivity. It should NOT be taken on a mobile device. 4. All electronic devices and reference material besides those permitted must be removed from the exam environment. 5. Compatible web browsers include updated versions of Mozilla Firefox or Google Chrome. Any other browser may not display questions correctly. 6. Please be mindful we may access logs of your Canvas activity in the event of any discrepancy or concerns regarding breaches of integrity. 7. The content of this exam is not to be shared or distributed in any form. 1. (15 marks. Upload your handwritten answers and short justifications. Short justifications are worth half of the marks.) Suppose that {Xn}n≥0 is a Markov Chain (MC) with state space S = {0,1,2,3} and transition matrix P: 1/2 1/2 0 0 P=α 0 β 0.  0 α 0 β  0010 where 0 ≤ α ≤ 1 and 0 ≤ β ≤ 1. (a) Find all values of α and β so that the MC is irreducible, aperiodic and recurrent. (b) Let α = β = 1/2. Find the stationary distribution of the MC. (c) Letα=β=1/2. FindthelimitofPn asn→∞. (d) For n ≥ 0, define the probability distribution of Xn by π(n) = ��P(Xn = 0), P(Xn = 1), P(Xn = 2), P(Xn = 3)��. Letα=β=1/2. Specifyπ(0) sothat,foralln≥0,Xn hasthesame probability distribution, i.e., π(n) = π(n−1) = ... = π(0). (e) Suppose that a system is modelled by the MC {Xn}n≥0 with α = β = 1/2, and a gambler plays against a machine based on the state of the system: in each game, he loses $1 if the system is at states 0, 1, 2 or he will win $5 if the system is at state 3. Find the gambler’s average winning in 100 such games. 2 2. (16 marks. Upload your handwritten answers and short justifications. Short justifications are worth half of the marks.) At time 0, a blood culture starts with one red cell. At the end of one minute, the red cell dies and is replaced by the following combinations with probabilities as indicated: 1 4 2 red cells 2 3  1 12 1 red, 1 white 2 white cells. Each red cell lives for one minute and then gives birth to offspring in the same way as the parent cell. Each white cell lives for one minute and dies without reproducing. The individual cells act independently of each other. Let Xn denote the number of red cells at time n + 1/2. (a) Explain why {Xn, n ≥ 0} is a branching process with X0 = 1 and the offspring distribution: f0 =1/12, f1 =2/3, f2 =1/4. (b) Calculate the probability of the event that there are no red cells at time 2.5. (c) Find the expected number of red cells n minutes after the culture began and the probability that the culture eventually dies out. (d) Suppose that, at time 0, a blood culture produces red cells according to a Poisson distribution with mean λ = 2 rather than just one red cell (other facts related to this question do not change). Find the expected number of red cells n minutes after the culture began and the probability that the culture eventually dies out. 3 3. (16 marks. Upload your handwritten answers and short justifications. Short justifications are worth half of the marks.) Motor vehicles arrive at a toll gate according to a Poisson process with rate λ = 2 vehicles per minute. The vehicles arriving at the gate belong to classes 1,2 or 3 with probabilities 1/2, 1/3 or 1/6, respectively. The drivers pay tolls of $1, $2 or $5 depending on the classes 1, 2, or 3 their vehicles belong to. (a) Find the probability that exactly $1 is collected in a period of 2 minutes. (b) Find the probability that the first toll collected is $1. (c) Find the probability that the waiting time between two vehicles that pay $5 is more than 10 minutes. (d) Find the mean and the variance of the amount in dollars collected in any given hour. 4 4. (13 marks. Upload your handwritten answers and short justifications. Short justifications are worth half of the marks.) A machine either works or does not work. The time for which it does work is exponentially distributed with mean λ (hours). When the machine does not work, the time it takes to repair is exponentially distributed with mean μ (hours). Denote by Xt the state (1 = works and 2 = does not work) of the machine at the time t. It is well-known that {Xt, t ≥ 0} is a homogeneous continuous Markov Chain with state space S = {1, 2}. (a) Show that the transition probability pij(h) satisfies the basic assumption: p11(h) = 1 − h/λ + o(h), p12(h) = h/λ + o(h), p21(h) = h/μ + o(h), p22(h) = 1 − h/μ + o(h). (b) Find Q-matrix and the transition matrix P of the jump chain. (c) Find stationary distribution and limit distribution of the MC. (d) When the machine does not work, it incurs a cost of $100 per hour. Find the minimum ratio of λ/μ so that the total cost is not more than $50 on average in a working day with 8 hours. 5