ST3SML
ASSIGNMENT
This assessment is due 12:00pm (mid-day) Friday 20th March 2020.
You are required to submit a hardcopy of your solutions AND a .R file
containing your annotated R code used to obtain your solutions.
Submit the hardcopy in the drop box in the Support Centre, JJT, and fill in a coursework
coversheet.
Submit your .R file on blackboard.
Note that is it not necessary to include code used for data manipulation and plotting in your
.R file. Plotting and data manipulation may be performed in any suitable software package.
Question
In linguistics, a phoneme is the smallest unit of sound that distinguishes one word from
another. This question concerns classification of the phonemes “aa”, as in the sound of the
vowel “a” in the word “dark”, and “ao” , as in the sound of the vowel “a” in the word
“water”, using samples from the TIMIT speech recognition database (TIMIT Acoustic-
Phonetic Continuous Speech Corpus, NTIS, US Dept of Commerce).
The data for this question were obtained from speech frames of phonemes taken from samples
of continuous speech by di↵erent speakers. The data are available on Blackboard as ASCII
files and consists of a column labelled “speaker”, a response column labelled “g” and 256
columns labelled “x.1” - “x.256”. Each row of “x.1” - “x.256” is a log-periodogram computed
from a speech frame of either “aa” or “ao” measured at 256 frequencies. A log-periodogram
is a widely used method for converting speech to a form suitable for speech recognition.
The training data is in file phoneme_train.txt and the test data is in phoneme_test.txt.
(a) Using the training data, draw line plots of a sample of 20 log-periodograms for the
phoneme “aa” and a sample of 20 log-periodograms for the phoneme “ao” against fre-
quencies 1 to 256, on the same graph. The log-periodogram values should be on the
y-axis and the frequencies on the x-axis.
Comment on the plots.
[5 marks]
1
(b) Perform logistic regression on the training data in order to predict a phoneme using its
log-periodogram and obtain the confusion matrix and training error rate. Do not include
an intercept term in your model.
Also compute the test error rate and a bootstrap 95% confidence interval for the test
error based on 1000 bootstrap estimates. You should include your code for computing
the bootstrap estimate in the hard copy of your solution.
Comment on your results.
[20 marks]
(c) Repeat part (b) using a QDA model and comment on the results. Also compare the
error rates with those obtained for the logistic regression model.
[20 marks]
(d) Here we investigate improving the test error rate of the logistic regression model in part
(b) by constructing a simple filter.
0 50 100 150 200 250
−0.4
−0.2
0.0
0.2
0.4
0.6
0.8
Frequency
β^
The figure above is a plot of the estimated parameters ˆ1, . . . , ˆ256 from the fitted logistic
model in part (b) with predictors x.1,. . .,x.256 against the frequencies 1, . . . , 256. The
2
rapid fluctuations seen in the plot indicates strong negative correlation between neigh-
bouring estimates and is due to the neighbouring frequencies in the speech frames being
highly positively correlated.
We now wish to construct a logistic regression model in which the parameter estimates
are forced to vary smoothly with frequency.
(i) Write R code to generate 13 natural cubic splines basis functions with knots uni-
formly placed over the integers 1, 2, . . . , 256, representing the frequencies, and con-
struct the 256⇥ 13 basis matrix B.
What are the elements in the last 6 rows of your matrix B?
Note: To avoid the elements in B becoming too large, you should rescale the fre-
quencies 1, 2, . . . , 256 to take values between 0 and 1 . The following code constructs
13 knots evenly placed between the rescaled frequencies.
knots<-quantile(1:256/256, probs=seq(0, 1, length.out=13))
[22 marks]
(ii) “Filter” the predictors x = (x.1,. . .,x.256) in the training data by computing x⇤ =
xB and fit a linear logistic regression model (without an intercept term) using x⇤
as your predictor variables.
What are the values of the parameter estimates ˆ⇤1 , . . . , ˆ
⇤
13 in your model?
[5 marks]
(iii) Construct the plot of ˆ1, . . . , ˆ256 from the model in part (b) against frequen-
cies 1, . . . , 256, as shown in the figure above, and superimpose on it a plot of the
smoothed parameter estimates ˆsmooth = Bˆ⇤, where ˆ⇤ is the vector
264 ˆ
⇤
1
...
ˆ⇤13
375 .
Comment on your smoothed curve.
[8 marks]
(iv) Obtain the training and test confusion matrices and error rates for your model
based on the filtered predictors.
[15 marks]
(e) Compare the error rates obtained in part (d) with those obtained for the logistic regres-
sion model in part (b) and write a summary of your findings.
[5 marks]
3