Bayesian Theory MATH11177 Friday 18th December 2020 1300-1500 † * † All students: you have an additional 1 hour to assemble and submit your PDF. Final submission deadline: 16:00. * Students with a Schedule of Adjustment: You are entitled to a further fixed additional 1 hour for this remote examination. Final submission deadline: 17:00 Attempt all questions Important instructions 1. Start each question on a new sheet of paper. 2. Number your sheets of paper to help you scan them in order. 3. Only write on one side of each piece of paper. 4. If you have rough work to do, simply include it within your overall answer – put brackets at the start and end of it if you want to highlight that it is rough work. MATH11177 Bayesian Theory 1 Distributional information • A random variable X has a Normal distribution Normal(µ, σ2), if its probability density function is 1√ 2piσ2 exp ( −(x− µ) 2 2σ2 ) −∞ < x <∞. E[X]=µ, V [X]=σ2. • A random variable X has a Gamma distribution Gamma(α, β), if its probability density function is βα Γ(α) xα−1 exp(−βx) x > 0. E[X]=αβ , V [X]= α β2 . • A random variable X has an Inverse Gamma distribution Γ−1(α, β), if its probability density function is βα Γ(α) x−(α+1) exp ( −β x ) x > 0. E[X]= βα−1 , for α > 1, and V [X]= β2 (α−1)2(α−2) , for α > 2. • A random variable X has an Exponential distribution Exp(λ), if its probability density function is λ exp(−λx) x > 0. E[X]= 1λ , V [X]= 1 λ2 . • A random variable X has a Beta distribution Beta(α, β), if its probability density function is Γ(α+ β) Γ(α)Γ(β) xα−1(1− x)β−1 x ∈ [0, 1]. E[X]= αα+β , V [X]= αβ (α+β)2(α+β+1) . • A random variable X has a Binomial distribution Bin(n, p), if its probability mass function is ( n x ) px(1− p)n−x x = 0, 1, 2, . . . , n E[X]=np, V [X]=np(1− p). • A random variable X has a Poisson distribution Poisson(λ), if its probability mass function is exp(−λ)λx x! x = 0, 1, 2, . . . E[X]=λ, V [X]=λ. • A random vector X1, . . . , Xk has a Multinomial distribution Mn(n, p1, . . . , pk), if its probability mass function is n! x1! . . . xk! px11 p x2 2 . . . p xk k xi = 0, 1, 2, . . . , n, k∑ i=1 xi = n E[Xi]=npi, V [Xi]=npi(1− pi), Cov[Xi, Xj ]=−npipj . [Please turn over] MATH11177 Bayesian Theory 2 Exam length is 2 hours. There are 4 questions. (1) (28 marks.) Earbuds produced at a new manufacturing plant have a probability, θ, of having defects. To estimate θ a study will be carried out and a Beta(α, β) distribution will be used a prior distribution for θ. However, three experts, Alice, Beth, and Charles, have different opinions about the hyperparameters, α and β, for the Beta prior. For 10 days in a row a quality control inspector samples earbuds in succession until they find a defective earbud. The numbers of earbuds they looked at before finding a defect for the 10 days are shown below: 6 26 38 36 13 23 11 8 14 49 thus a total of 224 earbuds were examined. We assume that the number of earbuds examined until a defect is found, Y , follows a Geometric(θ) distribution with pmf Pr(Y = k) = (1− θ)k−1θ, k = 1, 2, . . . (a) Before carrying out a Bayesian analysis, calculate the maximum likelihood estimate (mle) for θ. [2 marks] (b) Using a Beta(1,1) prior for θ, write the posterior mean as a weighted combination, wEprior(θ) + (1 − w)θˆ, of the prior mean, Eprior(θ), and the mle, θˆ, and report the weight w given to the prior. [4 marks] (c) Based on experience at other plants, Alice had the prior opinion that the mean value of θ should be 0.03 with a coefficient of variation of 20%. What is the corresponding distribution? [6 marks] (d) Beth had a different prior for θ, namely, Beta(4,96). Given the above results, what is the posterior distribution for θ? [4 marks] (e) Charles, who had yet a third prior, ended up with the following posterior for θ: Beta(12, 313). Charles had a zero-one loss function for θ. What is the corresponding Bayes Estimator, θˆBE? [5 marks] (f) Using Charles’ results, calculate a normal approximation to the posterior distribution, reporting numerical values for the mean and the variance. [7 marks] [Please turn over] MATH11177 Bayesian Theory 3 (2) (24 marks.) The following problems are referring to Problem 1. (a) One way to characterize the opinions of Alice, Beth and Charles is to consider a discrete space for θ, namely Θ= (0.02, 0.03, 0.04) and postulate the following three hypotheses: HA : θ = 0.03 HB : θ = 0.04 HC : θ = 0.02 where the subscripts A, B, and C denote Alice, Beth, and Charles. Given equal probability priors to each hypothesis, Pr(HA) = Pr(HB) = Pr(HC) = 1/3, and using the observations below: 6 26 38 36 13 23 11 8 14 49 Calculate the posterior probabilities for each hypothesis. [10 marks] (b) Alice and Beth were certain that θ ≤ 0.06 and formulated two other hypotheses with this constraint in mind: H0 : 0.000 < θ ≤ 0.035 H1 : 0.035 < θ ≤ 0.060 where H0 reflected Alice’s opinion and H1 reflected Beth’s opinion. They agreed on the following right-truncated Beta distribution, Beta(α=4, β= 124), namely, pi(θ) = Γ(4 + 124) Γ(4)Γ(124) θ4−1(1− θ)124−1 × c× I(0 < θ ≤ 0.06) where c=1.05223. Figure 1 shows the pdf for the truncated Beta. With this prior on θ, the induced prior probability for H0 is 0.687, and then the prior for H1 is 0.313. Given the above n=10 observations: (i) Calculate Pr(H0|y). [10 marks] (ii) Calculate Bayes Factors BF01 and BF10. [2 marks] (iii) State conclusions about these hypotheses. [2 marks] You might use some of the following results: Dist’n Beta(14,338) Beta(14,348) Beta(10,146.4) Pr(θ ≤ 0.035) 0.3474791 0.3861674 0.04753540 Pr(θ ≤ 0.060) 0.9621810 0.9713330 0.45649210 [Please turn over] MATH11177 Bayesian Theory 4 0.01 0.02 0.03 0.04 0.05 0.06 5 10 15 20 25 30 Right−truncated Beta Prior θ Figure 1: Beta(4,124) right truncated at 0.06. [Please turn over] MATH11177 Bayesian Theory 5 (3) (22 marks.) (a) Describe the inverse probability integral transform procedure for generating a sample from a Weibull(α, β) distribution, where the pdf is the following: f(θ|α, β) = α β ( θ β )α−1 exp [ − ( θ β )α] , 0 < θ. [7 marks] (b) The number of beetles, y, on a randomly selected square meter of the trunk of a spruce trees is assumed to follow a Poisson(θ). An entomologist thinks that the average number of beetles is approximately 5 with a CV of 0.50 and uses a Lognormal(log(5), 0.4722) prior for θ, namely, pi(θ) = 1 θ 1√ 2pi ∗ 0.4722 exp [ −(ln(θ)− ln(5)) 2 2 ∗ 0.4722 ] . Write down the formula to calculate the marginal pdf for a sample of beetle counts, y = (y1, . . . , yn)—but do not try to solve. Describe an importance sampling algorithm to estimate the marginal pdf, where the envelope or importance sampler is Gamma(4, 0.8). Make the description as explicit as possible including expressions for the densities and weights. [10 marks] (c) Referring to the previous problem (3b), explain how to use SIR algorithm, with the same Gamma sampler, to generate a sample from the posterior for θ. [4 marks] (d) Explain how to use the resulting SIR sample in the previous problem (Problem 3c) to construct an 80% symmetric credible interval for θ. [1 mark] [Please turn over] MATH11177 Bayesian Theory 6 (4) (26 marks.) A study to estimate the survival of Atlantic puffins (a type of sea bird) was undertaken. A random sample n=20 puffins had radio collars placed on them at the beginning of June. The following numbers were recorded: • number that had died by 30 June (y1=8) • number alive on 30 June but dead by 31 July (y2=3) • number alive on 31 July but dead by 31 August (y3=7) • those still alive on 31 August (y4=2) The probability of surviving the first month is φ1, and the subsequent months all have an equal probability probability of surviving φ2. Assuming independence between the puffins, the probability distribution for (y1, y2, y3, y4) is multinomial: (y1, y2, y3, y4)|φ1, φ2 ∼ Multinomial ( n, (1− φ1), φ1(1− φ2), φ1φ2(1− φ2), φ1φ22 ) Independent Beta distribution priors are used for the two parameters: φ1 ∼ Beta(a1, b1) φ2 ∼ Beta(a2, b2) (a) Write down the posterior distribution for φ1 and φ2 up to a constant of proportionality in the most concise form possible. [6 marks] (b) Letting a1=b1=a2=b2=1, and including the above observed values for y, what are the marginal posterior distributions for φ1 and φ2? [6 marks] (c) Write down the steps of the implementation of a single-update-at-a-time Metropolis-Hastings sampler to generate a sample from the joint posterior distribution of φ1 and φ2. Use independent symmetric proposal distributions for both φ1 and φ2. [10 marks] (d) Name at least two diagnostics for assessing the quality of the MCMC sample. [4 marks] [End of Paper]
欢迎咨询51作业君
