Drs. J. Senk & P. Wijeratne: Applied Machine Learning @ University of Sussex – Spring 2025
Coursework Assignment
1 Assignment Overview
This assignment will involve you designing, building, testing and critiquing systems for two
applied machine learning tasks.
1. Task 1 (50%): A system for performing spam detection, aka. classifying spam from non-
spam in text.
2. Task 2 (50%): A system for performing face alignment, aka. locating facial landmarks in
images of people.
This assignment is worth 100% of the grade for this module. It is designed to ensure you
can demonstrate achieving the learning outcomes for this module, which are to:
•Determine the applicability of different machine learning models to data found in real-
world applications.
•Propose designs for simple systems, including appropriate pre-processing, to solve practical
problems using machine learning.
•Implement and document a computer program that learns and applies machine learning
models to realistic data.
•Critically evaluate the efficacy of proposed systems and appropriately communicate this
analysis.
2 What to hand in?
1. A report that comprises amaximumof 10 pages and 3000 words, including captions but
excluding references. We expect several pictures, diagrams and flowcharts to be included.
Please only use the .zip archive format for your submission.
The report should be written in two sections, one for each task. For each task, you
should cover the following points. More detail is provided in Sections 3 and 4 below.
•A summary and justification for all the steps in your system, including preprocessing,
choice of features and prediction model. Explaining the system diagrammatically is
very welcome.
•Results of your experiments. This should include some discussion of qualitative (ex-
ample based) and quantitative (number based) comparisons between different ap-
proaches that you have experimented with.
1
Drs. J. Senk & P. Wijeratne: Applied Machine Learning @ University of Sussex – Spring 2025
•Examples of failure cases in your system and a critical analysis of these, identifying
potential biases of your approach.
2. Either .ipynb files or .py files containing annotated code for all data preprocessing, model
training and testing.
3. For Task 1: A csv file that contains the predicted labels on thetestset of text, found in
the csv file (spamdetectiontestdata.csv) here. Youmustuse the provided “saveascsv”
function in the Colab worksheet to process an array of shape (number
testdata, 1) to a
csv file. Please make sure you run this on the right data and submit in the correct format
to avoid losing marks.
4. For Task 2: A csv file that contains the face landmark positions on thetestset of images,
found in the compressed numpy file (face
alignmenttestimages.npz) here. Youmustuse
the provided “saveascsv” function in the Colab worksheet to process an array of shape
(numbertestimage, numberpoints, 2) to a csv file. Please make sure you run this on the
right data and submit in the correct format to avoid losing marks.
Note! Do not reorder the test data or it will not match up with the test labels / points!
3 Task 1: Spam Detection
3.1 Mark allocation
25 marks will be awarded for writing and presentation and 25 for coding and data analysis.
10 MarksAccuracy and robustness of spam detection
These marks are allocated based on the performance of the spam detection method. This
will be evaluated on the held out test set. The test data (without labels) are provided in
the csv file (spam
detectiontestdata.csv) here and the error on the predicted labels will
be calculated after submission. Marks will be awarded for average accuracy and robustness
(based on the confusion matrix of your predictions). Note that only we have the test labels!
25 MarksOutline of methods employed
Justifying and explaining design decisions for the spam detection. This does not have to be
in depth, and we do not expect you to regurgitate the contents of the lecture notes/papers.
You should state clearly:
•any text pre-processing steps you have used, and why.
•what text features/representation you have used, briefly describe how they were cal-
culated, and why you chose them.
•what predictions methods you have use; what ML task this corresponds to, the type
of model that you have used, and the loss function that your system is trained with.
•design/parameter decisions should be explained and justified.
For top marks, you should clearly demonstrate a creative and methodical approach for
designing your system, drawing ideas from different sources and critically evaluating your
choices. Explaining using diagrams and/or flowcharts is very welcome.
2
Drs. J. Senk & P. Wijeratne: Applied Machine Learning @ University of Sussex – Spring 2025
15 MarksAnalysing results and failure cases
Critically evaluate the results produced by your system on validation data. You should
include quantitative (number based) and qualitative (example based) comparisons between
different approaches that you have tried (on a held-out validation set).
Quantitative measures include calculating the confusion matrix of your predictions. Please
note that we are interested in your final prediction results, rather than how the cost
function changes during training. Please explicitly define any evaluation metrics and
ensure they are appropriate for the task.
3.2 Most important links
Contentsfiletypelinks
Training text and label datacsv file (spamdetectiontrainingdata.csv)link
Test text data (without labels)csv file (spamdetectiontestdata.csv)link
Colab worksheet with some useful functionsColab worksheetlink
3.3 Where to start?
Text classiciation is covered in lecture 14, so that’s a good place to look for information. Other
lectures (e.g., lecture 15) are also helpful.
We have included a very basic Colab worksheet illustrating how to load the data and print
random text examples based on their labels. An example print-out is shown in Figure 1.
The simplest approach would be to treat this as aclassification problem, where given text
data you want to predict the whether or not it is spam.
To follow this approach you will need to consider what natural language pre-processing steps
are necessary to obtain suitable features for your predictive model.
Figure 1:Example of non-spam text (label == 0) in the training dataset
(spamdetectiontrainingdata.csv).
3
Drs. J. Senk & P. Wijeratne: Applied Machine Learning @ University of Sussex – Spring 2025
4 Task 2: Face Alignment
4.1 Mark allocation
25 marks will be awarded for writing and presentation and 25 for coding and data analysis.
10 MarksAccuracy and robustness of face alignment
These marks are allocated based on the performance of the face alignment method. This
will be evaluated on the held out test set. The test images, without annotations are
provided in the compressed numpy file (face
alignmenttestimages.npz) here and the error
on the predicted points will be calculated after submission. Marks will be awarded for
average accuracy and robustness (% of images with error below a certain threshold). Note
that only we have the test points!
25 MarksOutline of methods employed
Justifying and explaining design decisions for the landmark finding. This does not have to
be in depth, and we do not expect you to regurgitate the contents of the lecture notes/-
papers. You should state clearly:
•any image pre-processing steps you have used, and why.
•what image features/representation you have used, briefly describe how they were
calculated, and why you chose them.
•what predictions methods you have use; what ML task this corresponds to, the loss
function that your system is trained with, and a description of any regularisation that
you may have used.
•design/parameter decisions should be explained and justified.
For top marks, you should clearly demonstrate a creative and methodical approach for
designing your system, drawing ideas from different sources and critically evaluating your
choices. Explaining using diagrams and/or flowcharts is very welcome.
15 MarksAnalysing results and failure cases
Critically evaluate the results produced by your system on validation data. You should
include quantitative (number based) and qualitative (example based) comparisons between
different approaches that you have tried (on a held-out validation set).
Quantitative measures include measuring the cumulative error distribution (see lecture
slides) or using boxplots or other plots to compare methods. Please note that we are
interested in your final prediction results, rather than how the cost function changes during
training. Please explicitly define any evaluation metrics and ensure they are appropriate
for the task.
4.2 Most important links
Contentsfiletypelinks
Training images and pointscompressednumpyarray
(facealignmenttrainingimages.npz)
link
Test images (without points)compressednumpyfile
(facealignmenttestimages.npz)
link
Colab worksheet with some useful functionsColab worksheetlink
4
Drs. J. Senk & P. Wijeratne: Applied Machine Learning @ University of Sussex – Spring 2025
4.3 Where to start?
Face alignment is covered in lecture 8, so that’s a good place to look for information. Other
lectures (e.g., lecture 7) are also helpful.
We have included a very basic Colab worksheet illustrating how to load the data and visualise
the points on the face. A visualisation of the average face and points across all training images
is given in Figure 2.
The simplest approach would be to treat this as either a regular or a cascadedregression
problem, where given an image you want to predict the set of continuous landmark coordinate
locations.
To follow this approach you will need to consider what image features are helpful to predict
the landmarks and what pre-processing is required on the data. Although you could directly
use the flattened image as input, this will not be the optimal data representation for this task.
A better representation would be to describe a set of locations, either evenly spaced across
the image, or in some more useful pattern (think about where in the image you might want to
calculate more information) using a feature descriptor, such as Scale-Invariant Feature Transform
(SIFT). These descriptions can then be concatenated together and used as input into a linear
regression model. Note that youdo notneed to use the keypoint detection process for this task
- rather the descriptors should be computed at defined locations (hint: look at sift.compute()
or similar) to create a representation of the image that is comparable across the dataset.
Figure 2: Illustration of the0-indexed(counting from 0 as you would in Python) locations of
the points on the average face. For example, if we wanted to find the nose, that’s index 2 so we
would look up points[2,:], which would give you the x and y coordinates.
5
Drs. J. Senk & P. Wijeratne: Applied Machine Learning @ University of Sussex – Spring 2025
5 General Points on the report
•Read things! Provide references to anything you find useful. You can take figures from
other works as long as you reference them appropriately.
•Diagrams, flowcharts and pictures are very welcome! Make sure you label them properly
and refer to them from the text.
•All plots should have correctly labelled axis and the font sizes must be readable in A4 page
format.
•All figures (including plots) should have descriptive captions.
6 Notes on using Colab
Either you can complete this project using Google Colab, which gives you a few hours of comput-
ing time completely free of charge, or you can use your personal/lab machine. The lab machines
are fairly powerful, so if you need more computing resource then try those!
If you are using Google Colab, try and familiarise yourself with some of its useful features.
To keep your saved models, preprocessed data etc. you can save it to Google drive following
the instructions here. You can also directly download a file you make in Colab using the code
below:
from google.colab import files
files.download(filename)
If you refactor code into extra .py files, these should be stored in your google drive as well,
or on Box such that they are easy to load into your Colab worksheet.
7 What software functionality can I use?
You arenotallowed to use generative AI tools (e.g., ChatGPT, Deepseek, etc.) to solve these
tasks or write your report.
You arenotallowed to use library functions that have been written to directly solve the
tasks you have been given, i.e. text classification and face alignment. Youcannot use the dlib
or mediapipe face alignment toolsor anything that provides similar functionality. Also,
face detection isnotrequired on this data.
You are free to use fundamental components and functions from libraries such as NLTK,
OpenCV, numpy, scipy, scikitlearn to solve this assignment, although you don’t have to. Here,
fundamental components refers to things like regression / classification models and pre-processing
/ feature extraction steps and other basic functionality.
In terms of tools and frameworks, it’s absolutely fine to use convolutional neural networks
(CNNs) or recurrent neural networks (RNNs) if you want to. The best packages would be either
TensorFlow (probably with Keras) or PyTorch. If you use such an approach you should be
sure to document how you chose the architecture and loss functions. A well justified and high
performing deep learning approach will receive equivalently high marks as if you had built it
any other way.
In terms of sourcing additional labelled data, this isnotallowed for this assignment. This
is because in real-world commercial projects you will typically have a finite dataset, and even if
6
Drs. J. Senk & P. Wijeratne: Applied Machine Learning @ University of Sussex – Spring 2025
there are possibly useful public datasets available, their license normally prohibits commercial
use. On the other hand data augmentation, which effectively synthesises additional training
examples from the labelled data that you have, is highly encouraged. If you use this, please try
and add some text or a flow-chart of this process in your report.
8 Top Tips for Success
•Refer to lecture slides and labs - all the information is in there to complete these tasks!
•Remember Occam’s razor: complexity should not be added unnecessarily. The more
complicated your system the more things to explain/justify etc.
•Start with a simple achievable goal and use that as a baseline to test against. Keep track
of early models/results to use as points of comparison.
•Remember that even if it doesn’t work well, having a go at both tasks is worthwhile. We’re
only looking for simple solutions and your explanation of your system design.
•Think about things that you have learned about in Applied Machine Learning. Dimen-
sionality reduction could be helpful. Overfitting and outliers may be an issue, and you
should consider using methods to minimise this.
•For Task 2: You don’t need to work at very high resolution to get accurate results. Partic-
ularly when doing initial tests, resize your images to a lower resolution images. Make sure
you also transform your training points so they are in the same geometry as the image
(i.e., if you half the size of the image along both axes, then make sure to half the (x,y)
position of the training points too). For your predicted points, make sure these are all at
the same resolution as the original images.
7
