Statistics 726 Assignment 4 Due: October 28, 2020, 11PM
Please submit a scanned document of your solutions on Canvas. The answers should be numbered
accordingly. The calculations should include all the necessary steps and/or reasoning that lead
to the final answer.
Please submit a .R file which includes all the R codes that you have written.
1. Consider the Gaussian AR(2) process given by
Xt − µ = φ1(Xt−1 − µ) + φ2(Xt−2 − µ) + εt,
where E[Xt] = µ, {εt}t∈Z ∼ IIDN(0, σ2).
(a) i. (3 marks) Derive an expression for the conditional log-likelihood function,
logL(µ, φ1, φ2, σ|X1, X2), based on X1, X2, . . . , Xn (n > 2).
ii. (3 marks) Using the result from part (a)i, demonstrate that maximizing the
conditional log-likelihood function with respect to µ, φ1 and φ2 is equivalent to
minimizing the least squares of the following regression model:
Y = Zβ + ε,
where Y ∈ Rn−2, Z ∈ R(n−2)×3, β = [µ(1 − φ1 − φ2]), φ1, φ2]> and ε =
[ε3, ε4, . . . , εn]
>. You are supposed to identify the representation of Y and Z.
iii. (5 marks) Write an R function named CSSar2 to compute the conditional sum
of squares estimates for µ, φ1, φ2 and σ
2.
The function definition is given below:
CSSar2 <- function(x)
x A numeric vector of observations.
iv. (1 mark) Generate n = 200 observations from the AR(2) process with µ = 5,
φ1 = 1.5, φ2 = −0.75 and σ2 = 2.
Hint: You may find arima.sim function in R to be useful.
v. (2 marks) For the generated observations in part (a)iv, compute the conditional
sum of squares estimates for µ, φ1, φ2 and σ
2 using the CSSar2 function defined
in part (a)iii. Compare these estimates with the output from Arima function in
the “forecast” package for R.
(b) i. (5 marks) Find the autocovariance function of {Xt}t∈Z at lag-0 (in terms of
ρ(1), ρ(2), φ1, φ2 and σ
2) and autocorrelation function at lags 1 and 2 (in terms
of φ1 and φ2).
ii. (3 marks) State the joint distribution of X1 and X2 in terms of γ(0) and ρ(1).
Note: You are not required to substitute the expressions for γ(0) and ρ(1).
1
Assignment 4, Semester 2, 2020
iii. (6 marks) Using the result from part (b)ii, derive an expression for the negative
log-likelihood function, −logL(µ, φ1, φ2, σ;X1, . . . , Xn).
Hint: For easy of calculations you may use the following notations: X2 =
(X1−µ,X2−µ)> and Cov(X2,X2) = σ2Γ2, where Γ2 ∈ R2×2. For non-singular
matrix A ∈ Rm×m and k ∈ R: |kA| = km|A| and (kA)−1 = k−1A−1, where |A|
denotes the determinant of A.
iv. (3 marks) Find an expression for the maximum likelihood estimator for σ2 (de-
note it by σˆ2ML).
v. (2 marks) By substituting σˆ2ML into the negative log-likelihood function, find the
concentrated negative log-likelihood function, −logL(µ, φ1, φ2;X1, X2, . . . , Xn)
in terms of σˆ2ML,Γ2 and n.
vi. (6 marks) Write an R function named neglogL to compute the concentrated
negative log-likelihood function obtained in part (b)v.
The function definition is given below:
neglogL <- function(par, x)
par A numeric vector of length 3 for the values of µ, φ1 and φ2.
x A numeric vector of observations.
vii. (6 marks) Using the function defined in parts (a)iii and (b)vi, write another
function in R named CSSMLar2 to compute the maximum likelihood estimates
for µ, φ1, φ2 and σ
2 using conditional sum of squares estimates for µ, φ1 and φ2
as the initial solutions.
The function definition is given below:
CSSMLar2 <- function(x)
x A numeric vector of observations.
Hint: Use the optim function in R to solve the non-linear concentrated negative
log-likelihood function.
viii. (3 marks) Using the observations generated in part (a)iv and CSSMLar2 function,
find the CSS-ML estimates for µ, φ1, φ2 and σ
2. Compare these estimates with
the output from Arima function in the “forecast” package for R by setting the
argument method to ‘‘CSS-ML’’.
Note: This question is only for pedagogical purposes. Do not assume that CSSar2 or
CSSMLar2 are better replacements for Arima or any other related functions defined in R.
2. Consider the Gaussian MA(1) process defined by
Xt = θεt−1 + εt,
where {εt}t∈Z ∼ IIDN(0, σ2).
Let X˜t−1t be the best linear predictor for Xt based on {Xt−1, . . . , X1}, Zt = Xt − X˜t−1t
and Z1 = X1. Define Var(Zt) = σ
2rt.
2
Assignment 4, Semester 2, 2020
(a) (5 marks) Using the result
Zt = Xt − Cov(Zt−1, Xt)
Var(Zt−1)
Zt−1,
show that
Zt = Xt − θ
rt−1
Zt−1,
rt = 1 + θ
2 − θ
2
rt−1
.
Hint: Refer to the results in slide 29 of the “ARIMA” handout.
(b) (1 mark) Write an expression for the negative log-likelihood function, −logL(θ, σ;X1, X2, . . . , Xn)
for n ≥ 1.
(c) (2 marks) Find an expression for the maximum likelihood estimator for σ2 (denote
it by σˆ2ML).
(d) (1 mark) By substituting σˆ2ML into the negative log-likelihood function, find the
concentrated negative log-likelihood function, −logL(θ;X1, X2, . . . , Xn).
(e) (6 marks) Write an R function named neglogL to compute the concentrated negative
log-likelihood function obtained in part (d).
The function definition is given below:
neglogL <- function(par, x)
par A numeric vector of length 1 for the value of θ.
x A numeric vector of observations.
(f) (6 marks) Using the function defined in part (e), write another function in R named
MLma1 to compute the maximum likelihood estimates for θ and σ2.
The function definition is given below:
MLma1 <- function(x)
x A numeric vector of observations.
Hint: Set the initial value of θ to zero (when using the optim function).
(g) (1 mark) Generate n = 200 observations from the MA(1) process with θ = 0.8 and
σ2 = 2.
(h) (3 marks) For the generated observations in part (g) compute the maximum likelihood
estimates for θ and σ2 using the MLma1 function defined in part (f). Compare these
estimates with the output from Arima function in the “forecast” package for R by
setting the argument method to ‘‘ML’’.
3
Assignment 4, Semester 2, 2020
3. Consider the Gaussian AR(2) process defined by
Xt = φ1Xt−1 + φ2Xt−2 + εt,
where {εt}t∈Z ∼ IIDN(0, σ2) and σ2 = 2.
(a) (3 marks) Let the two roots of the polynomial φ(z) = 1− φ1z − φ2z2 be written as
z = r[cos(ω)± isin(ω)],
where r = 3 and ω ∈ (−pi, pi). Choose a non-zero value for ω and calculate the values
of φ1 and φ2.
(b) (5 marks) Using the values of φ1 and φ2 computed in part (a) and σ
2, generate 2000
time series of length n = 50. For each series, find the maximum likelihood estimates
for φ1, φ2 and σ
2. Suppose they are denoted by {φˆ1,ts}2000ts=1, {φˆ2,ts}2000ts=1 and {σˆ2ts}2000ts=1
for φ1, φ2 and σ
2, respectively.
Hint: You may use Arima function in the “forecast” package for R.
(c) (1 mark) Compute the mean of {φˆ1,ts}2000ts=1 obtained in part (b).
(d) (1 mark) Repeat part (c) for {φˆ2,ts}2000ts=1 and {σˆ2ts}2000ts=1.
(e) (6 marks) Repeat parts (b)–(d) for n = 150, n = 250 and n = 500.
(f) (3 marks) Comment on the magnitude of the difference between mean of {φˆ1,ts}2000ts=1
and φ1 as n increases. Repeat this step for φ2 and σ
2.
Hint: You may set the seed of the random number generator using set.seed function in
R.
4. Consider the data set wmurders, which are the annual totals of women murdered (per
100,000 standard population) in the United States. The data set is available through
fpp2 package in R.
(a) (10 marks) Using time plots, ACF and PACF plots, suggest an appropriate ARIMA(p, d, q)
model for these data. Give reasons for your selection.
Note: You may consider applying a simple mathematical transformation to stabilize
the variance before fitting the models.
(b) (2 marks) Write the model you suggested in terms of the backward-shift operator.
(c) (4 marks) Fit the model using R and examine the residuals. Is the model satisfactory?
(d) (8 marks) Use the model fitted in part (c) to compute forecasts for the next 3 years
(use functions in R to perform this task). Manually compute these forecasts and
compare your results with those produced in R.
4
Assignment 4, Semester 2, 2020
(e) (4 marks) Does auto.arima function in R give the same model that you have chosen
in part (a)? If not, which model do you think is more appropriate? Give reasons for
your selection.
Total: 120 marks
5

欢迎咨询51作业君