Module Code: MATH5745M01
Q1.
(a) Suppose that A is a square n× n matrix. What do you understand by the phrase
“matrix A is not of full rank”? Outline some properties of a not full rank matrix.
How would you check whether the matrix A is not of full rank? [8 marks]
(b) Suppose that B is a symmetric matrix such that B > 0. Explain carefully what
the notation “B > 0” tells you about the matrix B. Outline some properties of
matrix B. [8 marks]
(c) Suppose you are told that a square n×n matrix C is an orthogonal matrix. What
does this tell you about matrixC? Outline some properties of matrix C. [6 marks]
(d) The spectral decomposition theorem states: “Any symmetric (n×n) matrix S can
be written as S = GDG′ where G is the matrix of standardized eigen-vectors of
S and D is a diagonal matrix of eigen-values of S.” Discuss the importance of the
spectral decomposition theorem to Multivariate Statistics. [4 marks]
(e) Why is the multivariate normal distribution important in Multivariate Analysis?
Discuss its usefulness and its limitations. [8 marks]
(f) The figure below shows the results of a cluster analysis of some data for 28 countries
based upon the values of two variables at the end of April 2020.
Explain to a non-statistician what the plot shows (you do not need to know what
the data is) and also explain the methodology used to construct the plot. Discuss
whether alternative methods might be suitable for producing a more informative
cluster analysis plot. [11 marks]
0
50
0
10
00
15
00
20
00
25
00
Data 30/4/2020: Average linkage − Euclidean distance
D
is
ta
nc
e
Br
a
zi
l
Po
la
nd
Ja
pa
n
S.
K
o
re
a
M
al
ay
sia Ira
n
R
om
an
ia
R
us
si
a
Fi
nl
an
d
N
et
he
rla
nd
s
Sw
e
de
n
Po
rtu
ga
l
UK
Fr
a
n
ce
G
er
m
a
ny
Is
ra
e
l
Ca
na
da
N
or
wa
y
Au
st
ria
D
en
m
ar
k
Sp
ai
n
Ic
el
an
d
Be
lg
iu
m
Ire
la
nd
Si
ng
ap
or
e
Ita
ly
US
A
Sw
itz
e
rla
nd
Page 1 of 13 Turn the page over
Module Code: MATH5745M01
Q2.
(a) The figure shows the contours of probability density functions from three bivariate
normal distributions with variables x1 and x2. They have the same mean vector
µ = (5, 5)′, but different structures of covariance matrix Σ.
(a)
x1
x2
0.01
0.02
0.03
0.04
0.05 0.06

0.07
0 2 4 6 8 10
0
2
4
6
8
10
(b)
x1
x2
0.01
0.02
0.03
0.04

0.05

0.06


0.0
7 0.08
0 2 4 6 8 10
0
2
4
6
8
10
(c)
x1
x2

0.05

0.1

0.15


0.2


0.3

0 2 4 6 8 10
0
2
4
6
8
10
Answer the following questions, and explain briefly your reasoning.
(i) Which figure has the highest correlation between x1 and x2? [2 marks]
(ii) Which figure shows independence between x1 and x2? [2 marks]
(iii) Which figure has the lowest generalised population covariance |Σ|? [2 marks]
(iv) What can you say about the relationship between x1 and x2 in each of figures
(a) and (b) and (c) above? [3 marks]
(v) If you are told that x2 = 6, what can you then say about the distribution of
x1 in each of figures (a) and (b) and (c) above? [6 marks]
(There is no need to try to estimate the mean and variance exactly.)
Page 2 of 13 Turn the page over
Module Code: MATH5745M01
Question Q2 continued:
(b) An anthropologist is interested in the physical characteristics of adult males in an
isolated tribe. He measures the total body height (x1), arm length (x2), and head
circumference (x3) of 20 adult males. The measurements are all in centimeters.
The sample mean vector x and sample covariance matrix S of the measurements
are given by
x =

154.661.7
53.4

 , S =

7.095 2.768 2.6952.768 5.589 3.495
2.695 3.495 2.673

 .
(i) Let X be the data matrix of size 20× 3 for the above example. Suppose the
anthropologist wishes to have the measurements of the data to be in inches
instead of centimeters. Using matrix operations, how would you define a new
data matrix Y with the measurements in inches? How would you obtain the
sample covariance matrix S(Y) of the data matrix Y if given S? Do you
expect the elements of S(Y) to be larger or smaller in magnitude than those
of S? Explain your reasoning. [4 marks]
(Note that there is no need to calculate the elements of S(Y). Also note that
one inch is equal to 2.54 centimeters.)
(ii) For the given sample mean vector x and the given sample covariance matrix
S above, what can you deduce about the three variables x1, x2 and x3 and
their inter-relationship? [4 marks]
(iii) Let R be the sample correlation matrix of X. How would you obtain R using
matrix operations if given S? What can you say when comparing R to the
sample correlation matrix R(Y) based upon the data matrix Y? Explain your
reasoning. [4 marks]
(c) The anthropologist is interested in the null hypotheses H0 : Σ = Σ0, where Σ is
the population covariance matrix of X and
Σ0 =

5 0 00 5 0
0 0 5

 .
(i) Looking at the structure of Σ0, what is being hypothesised by the anthropol-
ogist with regard to the dependencies between variables? Do you think this is
a plausible hypothesis? Explain your answer. [3 marks]
(ii) Suppose V is the maximum likelihood estimate of the covariance matrix of X.
How do you calculate V if given S? [1 mark]
(iii) From the above data, we have loge |Σ−10 V| = −2.64038, and trace(Σ−10 V) =
2.91783. Test the null hypothesis H0 at the 5% significance level. What do
you conclude? [4 marks]
(iv) If the null hypothesis were of the form Σ0 = vI, where I is a (3× 3) identity
matrix, and given the values in (c)(iii) above, write down your new test
statistic U in terms of v. What value of v makes U a minimum? [5 marks]
Page 3 of 13 Turn the page over
Module Code: MATH5745M01
Q3.
Morphology is the branch of biology that deals with the form (structure) of living organ-
isms. An expert measures the length (in cm) and weight (in hundred of grams) of 20
adult birds from the same species, but from two different sub-species (10 birds in each
sub-species). The data can be seen in the following figure, where the points are marked
differently to distinguish observations from sub-species 1 and sub-species 2.
40 42 44 46 48
4
6
8
10
12
14
Length (cm)
W
e
ig
ht
(1
00
gr
)
Sub−species 1
Sub−species 2
The sample mean for Variety 1 is y
1
= (45.4, 8.01)′ and for Variety 2 it is y
2
=
(43.0, 10.06)′. The pooled sample covariance matrix Sp and its inverse S
−1
p is given by:
Sp =
(
3.578 2.053
2.053 2.002
)
, S−1p =
(
0.6795 −0.6970
−0.6970 1.2145
)
.
(a) Describe the principle of linear discriminant analysis. [3 marks]
(b) Looking solely from the horizontal axis (Length) or vertical axis (Weight) in the
above figure, can you identify a clear separation between the two varieties? Explain
briefly your reasoning. [2 marks]
(c) Find the discriminant function from the above data. What can you say about the
discriminant function line? Calculate the standardised coefficients of the discrimi-
nant function. [5 marks]
(d) Suppose the expert found two new observations. The first one is a bird with length
44 cm and weight 8.7 (in units of 100 grams). The second one is a bird with length
47 cm and weight 9.0 (in units of 100 grams).
(i) Before performing any calculations, identify to which sub-species should each
new observation be classified. Explain briefly your reasoning. [3 marks]
(ii) Now write down the discriminant rule. Based on this rule, to which sub-species
should each new observation be classified? [4 marks]
Page 4 of 13 Turn the page over
Module Code: MATH5745M01
Question Q3 continued:
(e) Consider the following two plots each showing data from two different groups.
42 43 44 45 46 47 48
4
6
8
10
12
(a)
x1
x2
0.0 0.5 1.0 1.5 2.0 2.5 3.0
4.
0
4.
4
4.
8
5.
2
(b)
x1
x2
For each of the above two figures, answer these questions:
(i) Do you see a group separation in the figure?
(ii) Is linear discriminant analysis suitable for separating the groups? Explain your
reasoning.
[4 marks]
(f) Discuss how linear discriminant analysis is similar to or is different from the aims
of cluster analysis. [4 marks]
Page 5 of 13 Turn the page over
Module Code: MATH5745M01
Q4.
Ten students in the School of Mathematics took the same set of modules in Semester 1,
denoted M1, M2, M3, and M4. Their module marks are shown in the following table,
where the students are labelled by A-J.
Student M1 M2 M3 M4
A 60 57 65 56
B 65 63 63 48
C 58 56 64 57
D 67 58 60 66
E 65 49 60 50
F 52 52 54 47
G 57 50 60 54
H 55 60 62 53
I 73 63 66 61
J 67 55 59 51
The sample mean vector y and covariance matrix S of the data are given by
y =


61.9
56.3
61.3
54.3

 , S =


42.544 13.589 10.144 17.033
13.589 24.456 10.789 9.678
10.144 10.789 12.233 9.456
17.033 9.678 9.456 35.122

 .
(a) Explain the idea of principal component analysis. [3 marks]
(b) Consider the following edited output of analysis in R, where dat contains the
dataset above and “??” replaces a real number.
> eigen(cov(dat))
$values
[1] ?? 21.594106 18.779765 5.446926
$vectors
[,1] [,2] [,3] [,4]
[1,] 0.6785561 -0.55556959 0.4795013 -0.03134488
[2,] 0.3995959 -0.17495326 -0.7956614 -0.42028241
[3,] 0.2901608 0.01717662 -0.3320533 0.89735850
[4,] 0.5437751 0.81267383 0.1635297 -0.13087369
(i) What is the trace of S? What does this trace represent? [2 marks]
(ii) What is the value that is replaced by ?? in the above R output? [1 mark]
(iii) Calculate the cumulative proportion of variability of the principal components.
Giving reasons, suggest how many principal components should be considered.
[6 marks]
(iv) Explain briefly your assessment of the loadings of the principal components
you selected in part (b)(iii) above. What proportion of total variability in the
data do your chosen principal components represent? [4 marks]
Page 6 of 13 Turn the page over
Module Code: MATH5745M01
Question Q4 continued:
(c) The following figure shows the first two principal components from the above
data, where the points have been replaced by the student labels. Comments on
the patterns that you see in the figure, in light of your answer in part (b)(iv) above
and the data table above. [4 marks]
A
B
C
D
E
F
G
H
I
J
−15 −10 −5 0 5 10 15
−
10
−
5
0
5
10
z1
z2
(d) Suppose, hypothetically, the sample covariance matrix that you observe is in the
form
S⋆ =


250 10 10 10
10 10 8 8
10 8 10 8
10 8 8 10

 .
What potential problem could arise in the interpretation of the results of the analysis
using S⋆? What solution do you recommend to deal with the problem? Explain
your reasoning. Would your solution give the same results as using S⋆? [3 marks]
(e) Explain briefly the differences and similarities between the aims of principal com-
ponent analysis and those of factor analysis. [2 marks]
Page 7 of 13 Turn the page over
Module Code: MATH5745M01
Q5.
An expert in modern language conducted an experiment in which the time (in millisec-
onds) of pronunciation of two different syllables, denoted x and y, was measured in three
different contexts from 12 unrelated individuals. The expert believes that both syllables
are highly correlated in terms of the time to pronounce them across different contexts.
Some of the data are shown in the following table:
x1 x2 x3 y1 y2 y3
28 29 37 30 32 42
28 24 28 26 34 35
28 29 29 33 32 27
27 25 25 25 24 29
...
...
...
...
...
...
The overall sample covariance matrix S can be partitioned as follows:
S =
(
Sxx Sxy
Syx Syy
)
.
The diagonal elements of S are given by (4.99, 12.73, 14.82, 12.45, 13.12, 27.42).
The corresponding sample correlation matrix Rxy between x variables and y variables is
given by
Rxy =


y1 y2 y3
x1 0.633 0.600 0.193
x2 0.390 0.442 −0.039
x3 0.438 0.416 0.451

.
(a) Explain what is the aim of canonical correlation analysis. Why is it done? [2
marks]
(b) Looking at the correlation matrix Rxy (and without doing any calculation), is the
expert’s belief likely? Explain your reasoning. [2 marks]
(c) Consider the following matrix
Mx = S
−1
xxSxyS
−1
yy Syx,
with eigenvalues and eigenvectors (as an output from R) satisfying:
Page 8 of 13 Turn the page over
Module Code: MATH5745M01
Question Q5(c) continued:
> eigen(mx)
$values
[1] 0.721597936 0.231768804 0.002894897
$vectors
[,1] [,2] [,3]
[1,] -0.9084010 -0.3187745 -0.8398968
[2,] -0.1633495 -0.6918638 0.4608839
[3,] -0.3848694 0.6478482 0.2866345
(i) What is the largest canonical correlation, denoted r1, between x’s and y’s? [1
mark]
(ii) What can be inferred when you compare r1 to the elements of Rxy? What
could be the reason? Explain your answer. [2 marks]
(d) Consider now the matrix
My = S
−1
yy SyxS
−1
xxSxy
with eigenvectors (as output from R) satisfying:
$vectors
[,1] [,2] [,3]
[1,] -0.8744107 0.2947793 -0.72690136
[2,] -0.3866843 -0.7338849 0.68363101
[3,] -0.2930549 0.6119789 0.06529217
What are the eigenvalues of My? Explain briefly your reasoning. [2 marks]
(e) (i) Which variables substantially contribute to the largest canonical correlation?
Explain briefly your reasoning. [5 marks]
(ii) Are the pair of canonical covariates you have considered in part (e)(i) above
independent? Explain briefly your answer. [2 marks]
(f) Test whether all correlations between x’s and y’s are zero at the 5% significance
level. Is the expert’s belief justified? [5 marks]
(g) If there was just one y variable and just one x variable, what would the matrices
Mx andMy correspond to? What do the eigen-vectors ofMx andMy now equal?
Explain your answer. [4 marks]
Page 9 of 13 Turn the page over
Module Code: MATH5745M01
Normal Distribution Function Tables
The first table gives
Φ(x) =
1√
2pi
∫ x
−∞
e−
1
2
t2dt
and this corresponds to the shaded area in the
figure to the right. Φ(x) is the probability that
a random variable, normally distributed with zero
mean and unit variance, will be less than or equal
to x. When x < 0 use Φ(x) = 1−Φ(−x), as the
normal distribution with mean zero is symmetric
about zero. To interpolate, use the formula
Φ(x) ≈ Φ(x1) + x− x1
x2 − x1
(
Φ(x2)− Φ(x1)
) −3 −2 −1 0 1 2 3
0.
0
0.
1
0.
2
0.
3
0.
4
x
Table 1
x Φ(x) x Φ(x) x Φ(x) x Φ(x) x Φ(x) x Φ(x)
0.00 0.5000 0.50 0.6915 1.00 0.8413 1.50 0.9332 2.00 0.9772 2.50 0.9938
0.05 0.5199 0.55 0.7088 1.05 0.8531 1.55 0.9394 2.05 0.9798 2.55 0.9946
0.10 0.5398 0.60 0.7257 1.10 0.8643 1.60 0.9452 2.10 0.9821 2.60 0.9953
0.15 0.5596 0.65 0.7422 1.15 0.8749 1.65 0.9505 2.15 0.9842 2.65 0.9960
0.20 0.5793 0.70 0.7580 1.20 0.8849 1.70 0.9554 2.20 0.9861 2.70 0.9965
0.25 0.5987 0.75 0.7734 1.25 0.8944 1.75 0.9599 2.25 0.9878 2.75 0.9970
0.30 0.6179 0.80 0.7881 1.30 0.9032 1.80 0.9641 2.30 0.9893 2.80 0.9974
0.35 0.6368 0.85 0.8023 1.35 0.9115 1.85 0.9678 2.35 0.9906 2.85 0.9978
0.40 0.6554 0.90 0.8159 1.40 0.9192 1.90 0.9713 2.40 0.9918 2.90 0.9981
0.45 0.6736 0.95 0.8289 1.45 0.9265 1.95 0.9744 2.45 0.9929 2.95 0.9984
0.50 0.6915 1.00 0.8413 1.50 0.9332 2.00 0.9772 2.50 0.9938 3.00 0.9987
The inverse function Φ−1(p) is tabulated below for various values of p.
Table 2
p 0.900 0.950 0.975 0.990 0.995 0.999 0.9995
Φ−1(p) 1.2816 1.6449 1.9600 2.3263 2.5758 3.0902 3.2905
Page 10 of 13 Turn the page over
Module Code: MATH5745M01
Percentage Points of the χ2-Distribution
This table gives the percentage points χ2ν(P ) for
various values of P and degrees of freedom ν, as
indicated by the figure to the right.
If X is a variable distributed as χ2 with ν de-
grees of freedom, P/100 is the probability that
X ≥ χ2ν(P ).
For ν > 100,
√
2X is approximately normally dis-
tributed with mean
√
2ν − 1 and unit variance.
0 χ2ν(P )
P/100
Percentage points P
ν 10 5 2.5 1 0.5 0.1 0.05
1 2.706 3.841 5.024 6.635 7.879 10.828 12.116
2 4.605 5.991 7.378 9.210 10.597 13.816 15.202
3 6.251 7.815 9.348 11.345 12.838 16.266 17.730
4 7.779 9.488 11.143 13.277 14.860 18.467 19.997
5 9.236 11.070 12.833 15.086 16.750 20.515 22.105
6 10.645 12.592 14.449 16.812 18.548 22.458 24.103
7 12.017 14.067 16.013 18.475 20.278 24.322 26.018
8 13.362 15.507 17.535 20.090 21.955 26.124 27.868
9 14.684 16.919 19.023 21.666 23.589 27.877 29.666
10 15.987 18.307 20.483 23.209 25.188 29.588 31.420
11 17.275 19.675 21.920 24.725 26.757 31.264 33.137
12 18.549 21.026 23.337 26.217 28.300 32.909 34.821
13 19.812 22.362 24.736 27.688 29.819 34.528 36.478
14 21.064 23.685 26.119 29.141 31.319 36.123 38.109
15 22.307 24.996 27.488 30.578 32.801 37.697 39.719
16 23.542 26.296 28.845 32.000 34.267 39.252 41.308
17 24.769 27.587 30.191 33.409 35.718 40.790 42.879
18 25.989 28.869 31.526 34.805 37.156 42.312 44.434
19 27.204 30.144 32.852 36.191 38.582 43.820 45.973
20 28.412 31.410 34.170 37.566 39.997 45.315 47.498
25 34.382 37.652 40.646 44.314 46.928 52.620 54.947
30 40.256 43.773 46.979 50.892 53.672 59.703 62.162
40 51.805 55.758 59.342 63.691 66.766 73.402 76.095
50 63.167 67.505 71.420 76.154 79.490 86.661 89.561
80 96.578 101.879 106.629 112.329 116.321 124.839 128.261
Page 11 of 13 Turn the page over
Module Code: MATH5745M01
Percentage Points of the t-Distribution
This table gives the percentage points tν(P ) for
various values of P and degrees of freedom ν, as
indicated by the figure to the right.
The lower percentage points are given by sym-
metry as −tν(P ), and the probability that |t| ≥
tν(P ) is 2P/100.
The limiting distribution of t as ν → ∞ is the
normal distribution with zero mean and unit vari-
ance. 0 tν(P )
P/100
Percentage points P
ν 10 5 2.5 1 0.5 0.1 0.05
1 3.078 6.314 12.706 31.821 63.657 318.309 636.619
2 1.886 2.920 4.303 6.965 9.925 22.327 31.599
3 1.638 2.353 3.182 4.541 5.841 10.215 12.924
4 1.533 2.132 2.776 3.747 4.604 7.173 8.610
5 1.476 2.015 2.571 3.365 4.032 5.893 6.869
6 1.440 1.943 2.447 3.143 3.707 5.208 5.959
7 1.415 1.895 2.365 2.998 3.499 4.785 5.408
8 1.397 1.860 2.306 2.896 3.355 4.501 5.041
9 1.383 1.833 2.262 2.821 3.250 4.297 4.781
10 1.372 1.812 2.228 2.764 3.169 4.144 4.587
11 1.363 1.796 2.201 2.718 3.106 4.025 4.437
12 1.356 1.782 2.179 2.681 3.055 3.930 4.318
13 1.350 1.771 2.160 2.650 3.012 3.852 4.221
14 1.345 1.761 2.145 2.624 2.977 3.787 4.140
15 1.341 1.753 2.131 2.602 2.947 3.733 4.073
16 1.337 1.746 2.120 2.583 2.921 3.686 4.015
18 1.330 1.734 2.101 2.552 2.878 3.610 3.922
21 1.323 1.721 2.080 2.518 2.831 3.527 3.819
25 1.316 1.708 2.060 2.485 2.787 3.450 3.725
30 1.310 1.697 2.042 2.457 2.750 3.385 3.646
40 1.303 1.684 2.021 2.423 2.704 3.307 3.551
50 1.299 1.676 2.009 2.403 2.678 3.261 3.496
70 1.294 1.667 1.994 2.381 2.648 3.211 3.435
100 1.290 1.660 1.984 2.364 2.626 3.174 3.390
∞ 1.282 1.645 1.960 2.326 2.576 3.090 3.291
Page 12 of 13 Turn the page over
Module Code: MATH5745M01
5 Percent Points of the F -Distribution
This table gives the percentage points Fν1,ν2(P )
for P = 0.05 and degrees of freedom ν1, ν2, as
indicated by the figure to the right.
The lower percentage points, that is the values
F ′ν1,ν2(P ) such that the probability that F ≤
F ′ν1,ν2(P ) is equal to P/100, may be found us-
ing the formula
F ′ν1,ν2(P ) = 1/Fν2,ν1(P ) 0 F (P )
P/100
ν1
ν2 1 2 3 4 5 6 12 24 ∞
2 18.513 19.000 19.164 19.247 19.296 19.330 19.413 19.454 19.496
3 10.128 9.552 9.277 9.117 9.013 8.941 8.745 8.639 8.526
4 7.709 6.944 6.591 6.388 6.256 6.163 5.912 5.774 5.628
5 6.608 5.786 5.409 5.192 5.050 4.950 4.678 4.527 4.365
6 5.987 5.143 4.757 4.534 4.387 4.284 4.000 3.841 3.669
7 5.591 4.737 4.347 4.120 3.972 3.866 3.575 3.410 3.230
8 5.318 4.459 4.066 3.838 3.687 3.581 3.284 3.115 2.928
9 5.117 4.256 3.863 3.633 3.482 3.374 3.073 2.900 2.707
10 4.965 4.103 3.708 3.478 3.326 3.217 2.913 2.737 2.538
11 4.844 3.982 3.587 3.357 3.204 3.095 2.788 2.609 2.404
12 4.747 3.885 3.490 3.259 3.106 2.996 2.687 2.505 2.296
13 4.667 3.806 3.411 3.179 3.025 2.915 2.604 2.420 2.206
14 4.600 3.739 3.344 3.112 2.958 2.848 2.534 2.349 2.131
15 4.543 3.682 3.287 3.056 2.901 2.790 2.475 2.288 2.066
16 4.494 3.634 3.239 3.007 2.852 2.741 2.425 2.235 2.010
17 4.451 3.592 3.197 2.965 2.810 2.699 2.381 2.190 1.960
18 4.414 3.555 3.160 2.928 2.773 2.661 2.342 2.150 1.917
19 4.381 3.522 3.127 2.895 2.740 2.628 2.308 2.114 1.878
20 4.351 3.493 3.098 2.866 2.711 2.599 2.278 2.082 1.843
25 4.242 3.385 2.991 2.759 2.603 2.490 2.165 1.964 1.711
30 4.171 3.316 2.922 2.690 2.534 2.421 2.092 1.887 1.622
40 4.085 3.232 2.839 2.606 2.449 2.336 2.003 1.793 1.509
50 4.034 3.183 2.790 2.557 2.400 2.286 1.952 1.737 1.438
100 3.936 3.087 2.696 2.463 2.305 2.191 1.850 1.627 1.283
∞ 3.841 2.996 2.605 2.372 2.214 2.099 1.752 1.517 1.002
Page 13 of 13 End.