COMP4702/COMP7703 - Machine Learning
Homework 10 - Gaussian processes
Marcus Gallagher
Core Questions
1. Using the same data as the prac and the covariance function
k(xp,xq) = exp
(
− 1
2`2
‖xp − xq‖22
)
exp
(
− 2
`2
sin2
(
pi‖xp − xq‖2/P
))
,
produce a plot as in Question 6 of the prac with σn = 1, ` = 2 and P = 3. You may use any
implementation of Gaussian process regression.
2. Consider the covariance function
k(xp,xq) = (xp · xq + c)d
For each value of d = 1, d = 2, d = 3, plot 5 random draws (as in the figure on page 2 of the prac)
on the interval [−6, 6] from a Gaussian process prior with the above kernel when c = 5. Use any
implementation to sample from the Gaussian process prior.
3. Based on your answer to question 2 or otherwise, guess which function class the covariance function
in question 2 represents.
Extension Questions
4. Bayesian treatment of standard linear regression. Let X be an n×d matrix of training inputs
with corresponding regression targets represented in an n dimensional vector y. Let xi and yi be the
ith row and element of X and y respectively. Assume a model of the form.
y = f(x) +
f(x) = x ·w
∼ N (0, σ2n),
where is independent of w.
(a) What is the likelihood, p(y|X,w)? (Hint: What is the distribution of y if f(x) is a fixed
number?)
(b) Put a prior over w, w ∼ N (0, I), where I is the identity matrix. Show that the log posterior
over w is given by
log p(w|X,y) = − 1
2σ2n
(
y −Xw
)>(
y −Xw
)
− 1
2
‖w‖2 + C,
where C is a constant not depending on w. (Hint: start with Bayes’ rule)
1
(c) Does the maximum in w over the log posterior (the MAP) correspond to the solution obtained
by any other machine learning technique(s) that you are aware of? Which one(s)?
(d) This model (in both the prior and the posterior) is a Gaussian process. What is the covariance
function of the Gaussian process prior over f(x)?
2

欢迎咨询51作业君