Assignment 4 - Foundations of Machine Learning CSCI3151 - Dalhousie University

Q1 (30%)
Clustering
In this question we are going to explore two different clustering methods on the ​wine dataset
and evaluate it using two measures: one is an intrinsic measure (no labels), while the other one
makes use of the available labels.

a) Cluster the dataset using the k-Means clustering algorithm without using the class information
as part of the features. Experiment with different numbers of clusters ranging from 2 to 5. What
is the variability of the resulting clusters as a function of different initializations? Use the
Silhouette coefficient and Adjusted Rand Index as metrics for evaluation.
b) Cluster the dataset using DBSCAN without using the class information as part of the features.
Experiment with different values for the parameters Eps and minPts. It’s up to you how you deal
with any outliers, but make sure your criterion is properly justified. What is the variability of the
resulting clusters as a function of different initializations? Use the Silhouette coefficient and
Adjusted Rand Index as metrics for evaluation.
c) Summarize your findings and comparison after experimenting with these two clustering
methods.


Q2 (40%)
Convolutional neural networks
In this question you will construct a convolutional neural network to classify a large set of low
resolution images. Similarly to what you have done with A2, we would like you to describe the
behavior of the network as you modify certain parameters.

Use the CIFAR-100 dataset (available from Keras)

from keras.datasets import cifar100

(x_train_original, y_train_original), (x_test_original, y_test_original) =
cifar100.load_data(label_mode='fine')

a) Using two convolutional layers, explore the impact on different choices for the number of
nodes and filter sizes for the two layers. Summarize your observations.
b) For the hyperparameters for the layers that you determined in part (a), experiment with a
higher number of epochs. Summarize your observations.
c) Experiment with two more activation functions in the convolutional layers using the optimal
number of epochs from part (b). Are you able to improve the validation accuracy?
d) For the result in (c), calculate test accuracy for each class. Do you observe lower accuracy
for any specific class or classes? Do you observe higher accuracy for any specific class or
classes?


Q3 (30%)
Recurrent Neural Networks
In this question you will experiment with a simple recurrent neural network, where you will try to
model a sinusoidal function with noise, whose amplitude becomes larger and larger as the
independent variable ​t increases ( ). This function can be expressed in Python as 0 ≤ t ≤ N
x=(np.sin(0.02*t)+2*np.random.rand(N))*(t/N) . For N=5000 we have:


The idea is that you will train a recurrent neural network with points up to a certain value, ​Tp​.
This is all the training points will be . The length of the sequence provided to the network pt ≤ T
is a parameter that you can tune.
You will use the notebook provided for this question named “A4Q3.ipynb”
a) Complete the notebook provided to fulfill this task. You need to add at least one
simpleRNN layer​ and a proper output layer.
b) Starting with ​length = 4, ​discuss how different choices of the length of the sequence fed
to the network can have an impact on performance.
c) Discuss how different network architectures can have an impact on performance.

Note 1: for simplicity, ​don’t change ​the values of the variables ​N​ and ​Tp​.
Note 2: make sure you understand the code before addressing these questions.


Submitting the assignment
Note that you will have three separate Assignments 4 on Brightspace, i.e. one for each
question (A4-Q1, A4-Q2, and A4-Q3)
1. Your assignment as a single .ipynb file including your answers should be submitted for each
question before the deadline on Brightspace.
Use ​markdown​ syntax to format your answers.
2. Do not clear your notebook. Keep all the results in your notebook. (​NEW​)
3. You can submit multiple editions of your assignment. Only the last one will be marked. It is
recommended to upload a complete submission, even if you are still improving it, so that you
have something into the system if your computer fails for whatever reason.
4. IMPORTANT: PLEASE NAME YOUR PYTHON NOTEBOOK FILE AS:
--Assignment-N-Q.ipynb, for example
Soto-Axel-Assignment-2-1.ipynb (for the first question of the second assignment)
A penalty applies if the format is not correct.
5. The markers will enter your marks and their overall feedback on Brightspace. In case that
there is any important feedback, it will be given to you, but otherwise you would need to refer to
the model solutions.

Marking the assignment

Criteria and weights. Each criterion is marked by a letter grade. Overall mark is the
weighted average of the grade of each criterion.
For the experimental questions:
0.2 Clarity: All steps are clearly described. The origin of all code used is clearly. Markdown is
used effectively to format the answer to make it easier to read and grasp the main points. Links
have been added to all online resources used (markdown syntax is: [AnchorText](URL) ).
0.2 Justification: Parameter choices or processes are well justified.
0.2 Results: The results are complete. The results are presented in a manner that is easy to
understand. The answer is selective in the amount and diversity of the experimental results
presented.
Only key results that support the insights are presented. There is no need to present every
single experiment you carried out. Only the interesting results are presented, where the
behaviour of the ML model varies.
0.4 Insights: The insights obtained from the experimental results are clearly explained. The
insights are connected with the concepts discussed in the lectures.
The insights can also include statistical considerations (separate training-test data,
cross-validation, variance).Preliminary investigation of the statistical properties of the attributes
(e.g. histogram, mean, standard deviation) is included.