Extra Exercises Applied Stochastic Models, MATH360 Output None. Delivery None. This assignment will not be graded. You do not have to submit your solutions. It does not contribute to the final grade. Exercise 1. Let (λi, i ≥ 1) be a sequence strictly positive real numbers. Let E1, E2, . . . be a sequence of independent random variables such that Ei ∼ Exp(λi), for i ≥ 1. (i) For n ≥ 1, define Wn = min{E1, . . . , En}. Prove that Wn ∼ Exp(λ), where λ = λ1 + · · ·+ λn. (ii) For n ≥ 1, let In = argmin{E1, . . . , En} be the (random) index of the minimum of the exponential random variables E1, . . . , En (i.e. for i = 1, . . . , n, In = i if Wn = Ei). Prove that P(In = i) = λi/λ, for i = 1, . . . , n. (iii) Prove that Wn and In are independent. Exercise 2. Consider a device that completely fails when a cumulative effect of k ∈ N shocks occurs. Suppose that the shocks happens according to a Poisson process of parameter λ > 0 and let T be the (random) life time of the device. Find the distribution of T (i.e. what is the density function of T?) Exercise 3. Let (Xt, t ≥ 0) be a Poisson process of parameter 1. Define the continuous-time stochas- tic process (Yt, t ≥ 0) by letting Yt = Xλt, for t ≥ 0 and some λ > 0. Prove that (Yt, t ≥ 0) is a Poisson process of parameter λ. Exercise 4. Let (Xt, t ≥ 0) be a Poisson process of parameter λ > 0. For t, s ≥ 0, compute the covariance between Xt and Xt+s (i.e. compute E[(Xt − E[Xt])(Xt+s − E[Xt+s])]). 1 Exercise 5. Let (Xt, t ≥ 0) be a Poisson process of parameter λ > 0. Find the finite-dimensional distributions of (Xt, t ≥ 0), that is, compute P(Xt1 = x1, . . . , Xtn = xn), for n ∈ N, 0 ≤ t1 ≤ · · · ≤ tn and 0 ≤ x1 ≤ · · · ≤ xn such that x1, . . . , xn ∈ N0. Exercise 6. Let (Xt, t ≥ 0) be a continuous-time Markov chain with state space S = {1, 2, 3, 4} and Q-matrix Q = −4 2 1 1 0 −1 1 0 3 0 −5 2 0 0 0 0 . Explain the evolution of (Xt, t ≥ 0), that is, write down the holding rates and transi- tion probabilities of the underlying jump chain. Draw also a diagram to represent the behaviour of the continuous-time Markov chain. Exercise 7. Two-state machine. Consider a machine that can be up or down at any time. If the machine is up, it fails after an exponential time of parameter µ > 0. If it is down, it is repaired after an exponential time of parameter λ > 0. The successive up times are i.i.d. and so are the successive down times, and the up and down times are independent from each other. Suppose that we model the state of the machine as a continuous-time Markov chain (Xt, t ≥ 0) with state space S = {0, 1} by letting Xt = 1 if the machine is up at time t ≥ 0, and Xt = 0 if the machine is down at time t ≥ 0. Compute the Q-matrix Q = (qi,j)i,j∈S of (Xt, t ≥ 0). Exercise 8. Consider a machine shop that has three machines, which are identical and work (or fail) independently. Each machine has its own repairman, and they are repaired indepen- dently. Each machine can be either up or down. If a machine is up, its life time has an exponential distribution with parameter µ > 0. If a machine is down, the time it takes to repair it has an exponential distribution with parameter λ > 0. We model the number of machines that are working as a continuous-time Markov chain (Xt, t ≥ 0) by letting Xt be the number of machines that are working at time t ≥ 0. (i) Note that the state space of (Xt, t ≥ 0) is S = {0, 1, 2, 3}. Compute the Q-matrix Q = (qi,j)i,j∈S of (Xt, t ≥ 0). (ii) For i = 0, 1, 2, 3 and t ≥ 0, define xi(t) = P(Xt = i|X0 = 3). Use the the Kolmogorov forward equations for (Xt, t ≥ 0) to write down a system of differential equations satisfied by xi(t) for all i = 0, 1, 2, 3 and t > 0. 2 Exercise 9. Customers arrive at a store as a Poisson process of rate 2. At the door, two represen- tatives separately demonstrate the same product to anybody entering the store. Each demonstration takes a time which is exponentially distributed with parameter 1, and is independent of other demonstrations. After the demonstration the customer enters the store. When both representatives are busy, customers go directly into the store. If both representatives are free at time t = 0, show that the probability that both are busy at t > 0 is 2 5 − 2 3e −2t + 415e −5t. Hint: You do not want to count customers in the shop. What is your continuous-time Markov chain? Explain (informally) that the process described is indeed a continuous- time Markov chain. (20 pts) Exercise 10. Let (Xt, t ≥ 0) be a continuous-time Markov chain with state space S = {1, 2, 3, 4} and Q-matrix Q = −1 1/2 1/2 0 1/4 −1/2 0 1/4 1/6 0 −1/3 1/6 0 0 0 0 . What are the communicating classes? For each class, say if it is closed or not, and recurrent or transient. Hint: Recall Proposition 7, Definition 35 and Corollary 4 in the Lectures Notes. Exercise 11. Let (Xt, t ≥ 0) be a continuous-time Markov chain with state space S = N0 and Q-matrix Q = (qi,j)i,j∈N where the only non-zero off-diagonal entries of are given by qn,n+1 = 2n, for n ≥ 0, and qn,n−1 = 2n, for n ≥ 1. Suppose that we have proved that (Xt, t ≥ 0) does not explode in finite time. Show that (Xt, t ≥ 0) has an invariant distribution and deduce that (Xt, t ≥ 0) is positive recurrent. 3 Exercise 12. Consider a M/M/1 queue system (Xt, t ≥ 0), that is, a single-server queue in which customers arrive in according to a Poisson process of rate λ > 0 and where service times are independent identically exponentially distributed with parameter µ > 0; see Example 12 in Lecture notes for details. In particular, Xt denote the length of the queue at time t ≥ 0 including any customer being served. We assume that X0 = 0 and that λ < µ. (i) Find the invariant distribution of (Xt, t ≥ 0). (ii) Formulate the ergodic theorems for (Xt, t ≥ 0). Exercise 13. Consider a machine shop that has three machines, which are identical and work (or fail) independently. Each machine has its own repairman, and they are repaired indepen- dently. Each machine can be either up or down. If a machine is up, its life time has an exponential distribution with parameter µ > 0. If a machine is down, the time it takes to repair it has an exponential distribution with parameter λ > 0. We model the number of machines that are working as a continuous-time Markov chain (Xt, t ≥ 0) by letting Xt be the number of machines that are working at time t ≥ 0. Note that the state space of (Xt, t ≥ 0) is S = {0, 1, 2, 3}. Prove that (Xt, t ≥ 0) is positive recurrent. Hint: Write the Q-matrix of (Xt, t ≥ 0) and observe that (Xt, t ≥ 0) is irreducible. Observe also that (Xt, t ≥ 0) is not explosive because the state space is finite. You may want to use Proposition 12 in the Lecture notes. (This type of how-to-do instruction will NOT be provided in the exam: you are supposed to know it.) Exercise 14. Let (Xt, t ≥ 0) be a continuous-time Markov chain with state space S = {1, 2, 3} and Q-matrix Q = −1 1 00 −1 1 1 0 −1 . Find the invariant distribution of (Xt, t ≥ 0), that is, solve the equation ξQ = 0, for ξ = (ξ(1), ξ(2), ξ(3)) (i.e. component-wise (ξQ)i = 3∑ j=1 ξ(j)qj,i = 0, for i = 1, 2, 3). Remark. One can check that the detailed balance equations associated to Q only have the trivial solution ξ˜ = (0, 0, 0) which is not a probability distribution. Therefore, we have proven that the converse of Proposition 13 in the Lecture notes is false (i.e. there 4 are Q-matrices for which there are invariant distributions that do not satisfy the detailed balance equations). Exercise 15. Let (Bt, t ≥ 0) be a standard Brownian motion. For s, t ≥ 0, prove that (i) E[|Bt −Bs|] = √ 2|t−s| pi . (ii) E[|Bt −Bs|2] = |s− t|. Exercise 16. Let (Bt, t ≥ 0) be a standard Brownian motion. For s, t ≥ 0 such that s 6= t, prove that P(Bt > Bs) = 1/2. Exercise 17. Let (Bt, t ≥ 0) be a standard Brownian motion. Define the continuous-time stochastic process (Xt, t ≥ 0) by letting Xt = −Bt, for t ≥ 0. Prove that (Xt, t ≥ 0) is a standard Brownian motion. Exercise 18. Let (Bt, t ≥ 0) be a standard Brownian motion. Check, with justifications, whether or not each of the following continuous-time stochastic processes (Xt, t ≥ 0) is a standard Brownian motion. (i) Xt = B2t , for t ≥ 0. (ii) Xt = Bt +Bt2 , for t ≥ 0. (iii) For a constant c > 0, Xt = 1cBc2t, for t ≥ 0. Exercise 19. Let (Bt, t ≥ 0) be a standard Brownian motion. Let (Xt, t ≥ 0) be the continuous-time stochastic process defined by Xt = −6 + 2t+ 3Bt, for t ≥ 0. Calculate the following, (i) P(X4 < 12). (ii) P(eX4 < 12). Exercise 20. Let (Bt, t ≥ 0) be a standard Brownian motion. Let (Xt, t ≥ 0) be the continuous-time stochastic process defined by Xt = eBt− t 2 , for t ≥ 0. Show that (Xt, t ≥ 0) satisfies the stochastic integral equation Xt = 1 + ∫ t 0 XsdBs, t ≥ 0. 5
欢迎咨询51作业君
