CSCE 421 :: Machine Learning :: Texas A&M University :: Spring 2023

Homework 6 (HW-6)

Convolutional Neural Networks

In this assignment, you'll be coding up a convolutional neural network from scratch to classify images using PyTorch.

Instructions

Install PyTorch following the instructions here (https://pytorch.org/).

Install the torchinfo package (https://github.com/TylerYep/torchinfo) to visualize the network architecture and the number of parameters. The maximum number of parameters you are allowed to use for your network is 100,000.

You are required to complete the functions defined in the code blocks following each question. Fill out sections of the code marked "YOUR CODE HERE" .

You're free to add any number of methods within each class.

You may also add any number of additional code blocks that you deem necessary.

Once you've filled out your solutions, submit the notebook on Canvas.

Do NOT forget to type in your name and UIN at the beginning of the notebook.

Data Preparation

In [ ]:

In [ ]:

     !pip install torchinfo

 # Importing the libraries

import os

import torch

import torchvision

from torchvision.utils import make_grid

import numpy as np

In this assignment, we will use the Fashion-MNIST dataset. Fashion-MNIST is a dataset of Zalando's article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes.

Data

Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255.

Labels

Each training and test example is assigned to one of the following labels:

Fashion-MNIST is included in the

In [ ]:

In [ ]:

torchvision

library.

Label Description

    0 1 2 3 4 5 6 7 8 9

T-shirt/top Trouser Pullover Dress Coat Sandal Shirt Sneaker Bag Ankle boot

 from torchvision.datasets import FashionMNIST

from torchvision.transforms import Compose, ToTensor, Normalize

 # Transform to normalize the data and convert to a tensor

transform = Compose([ToTensor(), Normalize((0.5,), (0.5,))

])

# Download the data

dataset = FashionMNIST('MNIST_data/', download = True, train = True, transform = transform)

Data Exploration

Let's take a look at the classes in our dataset.

In [ ]:

 print(dataset.classes)

In [ ]:

 import matplotlib.pyplot as plt

def show_example(img, label):

print('Label: {} ({})'.format(dataset.classes[label], label)) plt.imshow(img.squeeze(), cmap='Greys_r')

plt.axis(False)

In [ ]:

In [ ]:

Creating Training and Validation Datasets

The split_indices function takes in the size of the entire dataset, n , the fraction of data to be used as validation set, val_frac , and the random seed and returns the indices of the data points to be added to the validation dataset.

Choose a suitable fraction for your validation set and experiment with the seed. Remember that the better your validation set, the higher the chances that your model would do well on the test set.

In [ ]:

 show_example(* dataset[20])

 show_example(* dataset[20000])

 def split_indices(n, val_frac, seed):

# Determine the size of the validation set n_val = int(val_frac * n) np.random.seed(seed)

# Create random permutation between 0 to n-1 idxs = np.random.permutation(n)

# Pick first n_val indices for validation set return idxs[n_val:], idxs[:n_val]

In [ ]:

 ######################

# YOUR CODE BELOW #

######################

val_frac = ## Set the fraction for the validation set rand_seed = ## Set the random seed

train_indices, val_indices = split_indices(len(dataset), val_frac, rand_seed) print("#samples in training set: {}".format(len(train_indices))) print("#samples in validation set: {}".format(len(val_indices)))

Next, we make use of the built-in dataloaders in PyTorch to create iterables of our our training and validation sets. This helps in avoiding fitting the whole dataset into memory and only loads a batch of the data that we can decide.

Set the batch_size depending on the hardware resource (GPU/CPU RAM) you are using for the assignment

In [ ]:

In [ ]:

In [ ]:

 from torch.utils.data.sampler import SubsetRandomSampler from torch.utils.data.dataloader import DataLoader

 ######################

# YOUR CODE BELOW # ######################

batch_size = ## Set the batch size

 # Training sampler and data loader

train_sampler = SubsetRandomSampler(train_indices) train_dl = DataLoader(dataset,

batch_size,

sampler= train_sampler)

# Validation sampler and data loader

val_sampler = SubsetRandomSampler(val_indices) val_dl = DataLoader(dataset,

batch_size,

sampler= val_sampler)

Plot images in a sample batch of data.

In [ ]:

In [ ]:

Building the Model

Grading: 20 pts for Build the Model section

They don't have to have the exact same model, e.g., the number of layers can be different, the parameters in the model

The very first parameter of the first layer has to be 1 nn.Conv2d(1, ....), because this is black-white dataset, otherwise -5pts

The very last parameter of the last layer has to be 10 nn.Linear(..., 10), otherwise -20pts.

 def show_batch(dl):

for images, labels in dl:

fig, ax = plt.subplots(figsize=(10,10))

ax.set_xticks([]); ax.set_yticks([])

ax.imshow(make_grid(images, 8).permute(1, 2, 0), cmap='Greys_r') break

 show_batch(train_dl)

using nn.Conv1d (if ever happens) -20pts. M P l2d10

Create your model by defining the network architecture in the ImageClassifierNet class. NOTE: The number of parameters in your network must be 100,000.

In [ ]:

In [ ]:

In [ ]:

The following code block prints your network architecture. It also shows the total number of parameters in your network (see Total params ).

NOTE: The total number of parameters in your model should be <= 100,000.

In [ ]:

##Check Model Requirements status

Below code cell is to check if your model meets the requirements specefied Run the cell to check the status of your model

≤

 # Import the libraries

import torch.nn as nn

import torch.nn.functional as F

from torchinfo import summary

 class ImageClassifierNet(nn.Module): def __init__(self, n_channels=3):

        super(ImageClassifierNet, self).__init__()

        ######################

        #   YOUR CODE HERE   #

        ######################

def forward(self, X): ###################### # YOUR CODE HERE # ######################

 model = ImageClassifierNet()

 summary(model, input_size=(batch_size, 1, 28, 28))

In [ ]:

 def check_model(model):

first_layer = list(model.children())[0]

last_layer = list(model.children())[- 1]

num_params = sum(p.numel() for p in model.parameters()) has_conv1d = False

has_maxpool2d = False

first_layer_status= False

last_layer_status = False

if isinstance(first_layer, nn.Sequential):

if isinstance(first_layer[0], nn.Conv2d) and first_layer[0].in_channels == 1:

first_layer_status= True else:

if isinstance(first_layer, nn.Conv2d) and first_layer.in_channels == 1: first_layer_status= True

if isinstance(last_layer, nn.Sequential):

if isinstance(last_layer[-1], nn.Linear) and last_layer[-1].out_features == 10:

last_layer_status = True else:

print(type(last_layer))

if isinstance(last_layer, nn.Linear) and last_layer.out_features == 10:

last_layer_status = True

for layer in model.modules():

if isinstance(layer, nn.Conv1d):

has_conv1d = True

if isinstance(layer, nn.MaxPool2d):

has_maxpool2d = True flag = False

if first_layer_status == False:

print("The very first parameter of the first layer is not 1 nn.Conv2d(1, ....)") flag = True

if last_layer_status == False:

print("The very last parameter of the last layer is not 10 nn.Linear(..., 10)") flag = True

if has_conv1d == True:

print("Using nn.Conv1d, which should not happen") flag = True

if has_maxpool2d == False: print("No nn.MaxPool2d") flag = True

if num_params > 100000:

print("Parameters > 100000, please make the network less complex to recieve full grade")

if flag == False:

print("The Network looks good")

check_model(model)

Enable training on a GPU

NOTE: This section is necessary if you're training your model on a GPU.

In [ ]:

 def get_default_device():

"""Use GPU if available, else CPU""" if torch.cuda.is_available():

return torch.device('cuda') else:

return torch.device('cpu')

def to_device(data, device):

"""Move tensor(s) to chosen device""" if isinstance(data, (list,tuple)):

return [to_device(x, device) for x in data] return data.to(device, non_blocking=True)

class DeviceDataLoader():

"""Wrap a dataloader to move data to a device""" def __init__(self, dl, device):

self.dl = dl self.device = device

def __iter__(self):

"""Yield a batch of data after moving it to device""" for b in self.dl:

yield to_device(b, self.device)

def __len__(self): """Number of batches""" return len(self.dl)

In [ ]:

Train the model

Grading: the two cells below 20 pts

Have to use both training set and validation set, if no validation set -15 pts

In training part, zero_grad() needs to be before backward(), and backwards() need to be before step(), wrong order or missing one -20

If any of zero_grad(), backward(), or step() happens in validation, -10pts

If MSE-related loss is used in loss_fn (the second cell), -15 pts

Complete the train_model function to train your model on a dataset. Tune your network architecture and hyperparameters on a validation set.

 device = get_default_device()

train_dl = DeviceDataLoader(train_dl, device)

val_dl = DeviceDataLoader(val_dl, device) to_device(model, device)

In [ ]:

 def train_model(n_epochs, model, train_dl, val_dl, loss_fn, opt_fn, lr): """

    Trains the model on a dataset.

    Args:

        n_epochs: number of epochs

        model: ImageClassifierNet object

        train_dl: training dataloader

        val_dl: validation dataloader

        loss_fn: the loss function

        opt_fn: the optimizer

        lr: learning rate

    Returns:

        The trained model.

        A tuple of (model, train_losses, val_losses, train_accuracies, val_accuracies)

"""

    # Record these values the end of each epoch

train_losses, val_losses, train_accuracies, val_accuracies = [], [], [], []

    ######################

    #   YOUR CODE HERE   #

    ######################

    # Validation process

val_loss = None val_accuracy = None ###################### # YOUR CODE HERE # ######################

    # Record the loss and metric

    ######################

    #   YOUR CODE HERE   #

    ######################

    # Print progress

if val_accuracy is not None:

print("Epoch {}/{}, train_loss: {:.4f}, val_loss: {:.4f}, train_accuracy: {:.4f}, va

.format(epoch+1, n_epochs, train_loss, val_loss, train_accuracy, val_accuracy) print("Epoch {}/{}, train_loss: {:.4f}, train_accuracy: {:.4f}"

.format(epoch+1, n_epochs, train_loss, train_accuracy)) return model, train_losses, val_losses, train_accuracies, val_accuracies

else:

l

)

In [ ]:

 ######################

# YOUR CODE BELOW # ######################

num_epochs = # Max number of training epochs loss_fn = # Define the loss function

opt_fn = # Select an optimizer lr = # Set the learning rate

Grading: results 20 pts

The final train_accuracy and val_accuracy both need to above 0.9 0.85 t0 0.9 -5pts

0.80 to 0.85 -15pts

below 0.80 -20 pts

In [ ]:

Plot loss and accuracy

In [ ]:

 history = train_model(num_epochs, model, train_dl, val_dl, loss_fn, opt_fn, lr) model, train_losses, val_losses, train_accuracies, val_accuracies = history

 def plot_accuracy(train_accuracies, val_accuracies): """Plot accuracies"""

plt.plot(train_accuracies, "-x") plt.plot(val_accuracies, "-o") plt.xlabel("Epoch")

    plt.ylabel("Accuracy")

    plt.legend(["Training", "Validation"])

    plt.title("Accuracy vs. No. of epochs")

Grading: 10 pts for the first figure Both training and validation need to go up It's okay if validation is not stable as long as it roughly goes up

Any of them apparently goes down -10pts

In [ ]:

 plot_accuracy(train_accuracies, val_accuracies)

In [ ]:

 def plot_losses(train_losses, val_losses): """Plot losses""" plt.plot(train_losses, "-x") plt.plot(val_losses, "-o") plt.xlabel("Epoch")

    plt.ylabel("Loss")

    plt.legend(["Training", "Validation"])

    plt.title("Loss vs. No. of Epochs")

Grading: 10 pts for the second figure Both training and validation need to go down Again, it's okay if validation is not stable or going down first then going up

If training apparently goes up, -10pts

If validation goes up from the beginning, -5pts

In [ ]:

Train a model on the entire dataset

We create a new model with same architecture and train it on the whole training data.

In [ ]:

NOTE: The next cell is necessary if you're training your new_model on a GPU. In [ ]:

In [ ]:

Grading: results 10 pts

The final Train_accuracy needs to above 0.9

0.85 to 0.9 -3pts 0.80 to 0.85 -5pts below 0.80 -10 pts

 plot_losses(train_losses, val_losses)

 new_model = ImageClassifierNet()

 to_device(new_model, device)

 indices, _ = split_indices(len(dataset), 0, rand_seed)

sampler = SubsetRandomSampler(indices)

dl = DataLoader(dataset, batch_size, sampler=sampler) dl = DeviceDataLoader(dl, device)

In [ ]:

 ######################

# YOUR CODE BELOW # ######################

num_epochs = # Max number of training epochs lr = # Set the learning rate

history = train_model(num_epochs, new_model, dl, [], loss_fn, opt_fn, lr) new_model = history[0]

Check Predictions

In [ ]:

 def view_prediction(img, label, probs, classes): """

Visualize predictions.

"""

probs = probs.cpu().numpy().squeeze()

fig, (ax1, ax2) = plt.subplots(figsize=(8,15), ncols=2) ax1.imshow(img.resize_(1, 28, 28).cpu().numpy().squeeze(), cmap='Greys_r') ax1.axis('off')

ax1.set_title('Actual: {}'.format(classes[label]))

ax2.barh(np.arange(10), probs)

ax2.set_aspect(0.1)

ax2.set_yticks(np.arange(10))

ax2.set_yticklabels(classes, size='small');

ax2.set_title('Predicted: probabilities')

ax2.set_xlim(0, 1.1)

    plt.tight_layout()

In [ ]:

 # Calculate the class probabilites (log softmax) for img

images = iter(dl)

for imgs, labels in images:

with torch.no_grad(): new_model.eval()

        # Calculate the class probabilites (log softmax) for img

probs = torch.nn.functional.softmax(new_model(imgs[0].unsqueeze(0)), dim=1) # Plot the image and probabilites

view_prediction(imgs[0], labels[0], probs, dataset.classes)

break

Save the model

In [ ]:

Compute accuracy on the test set

Here we load the test data pickle file provided.

You need to provide the path for the test_data pickle file and run the cells.

Note: Your score will be given based on your model's performance on a test data set hidden from you. But you can expect similar results from both the test data set provided to you ans the one hidden from you.

Grading: results 10 pts

The final Test_accuracy needs to above 0.9

0.85 to 0.9 -3pts 0.80 to 0.85 -5pts below 0.80 -10 pts

In [ ]:

In [ ]:

 torch.save(new_model, 'new_model')

 import pickle

 ######################

# YOUR Answer BELOW # ###################### test_dataset_file_path = "test_data.pickle"

with open(test_dataset_file_path, 'rb') as f: test_dataset = pickle.load(f)

In [ ]:

 test_dl = DataLoader(test_dataset, batch_size) test_dl = DeviceDataLoader(test_dl, device)

In [ ]:

 def evaluate(model, test_dl): with torch.no_grad():

model.eval() total_test_dl = 0 preds, labels = [], [] for xb, yb in test_dl:

            # Model output

y_pred = model(xb)

_, y_pred = torch.max(y_pred, dim=1) preds.extend(y_pred) labels.extend(yb)

total_test_dl += len(yb)

preds, labels = torch.tensor(preds), torch.tensor(labels) test_accuracy = torch.sum(preds == labels).item() / len(preds) return test_accuracy

In [ ]:

 print("Test Accuracy = {:.4f}".format(evaluate(new_model, test_dl)))