Homework 07: Spiking Neural Networks
As we've been discussing in our lectures, spiking neural networks are an alternative to more conventional artificial neural network architectures that make use of activation functions like ReLU. In the context of our study of biologically-inspired models, they are interesting because they mimic the function of biological neural networks. An important question is whether or not they lead to empirical improvements in performance. In this assignment you will explore this question.
For this assignment, record your responses to the following activities in the
README.md file in the homework07 folder of your assignments GitLab
repository and push your work by 11:59 PM Monday, December 2.
Activity 0: Branching
As discussed in class, each homework assignment must be completed in its own git branch; this will allow you to separate the work of each assignment and for you to use the merge request workflow.
To create a homework07 branch in your local repository, follow the
instructions below:
$ cd path/to/cse-40171-fa19-assignments # Go to assignments repository $ git remote add upstream https://gitlab.com/wscheirer/cse-40171-fa19-assignments # Switch back over to the main class repository $ git fetch upstream # Toggle the upstream branch $ git pull upstream master # Pull the files for homework07 $ git checkout -b homework07 # Create homework07 branch and check it out $ cd homework07 # Go into homework07 folder
Once these commands have been successfully performed, you are now ready to add, commit, and push any work required for this assignment.
Activity 1: Train a Spiking Neural Network for MNIST Digit Classification (25 Points)
After cloning the repository for this assignment, you will have access to a collection of classes and methods in the homework07/SpikingNeuralNet.py source file that implement a small spiking neural network in the PyTorch framework, along with helper code for data processing and analysis. In a nutshell, this implementation works by adding a time axis to the network so that it sees data within a temporal context. Activation functions are spikes that are raised past a pre-activation threshold. Pre-activation values fade if units are not excited enough.
Using the provided code, write a small wrapper script that can train a spiking neural network for the MNIST dataset, which can perform digit classification. Instantiate the model by using the SpikingNet() constructor. Set the number of time steps to 128. Train the network with the train_many_epochs() method.
Calculate the wall time for training a spiking neural network model. Record this value in your README.md file. Copy the output of your training run (train and test loss and accuracy) into your README.md as well.
Activity 2: Train a Feed Forward Neural Network for MNIST Digit Classification (25 Points)
Now let's compare the spiking neural network with a more typical feed forward architecture for the same task. Again using the provided code, write a small wrapper script that can train a feed forward neural network for the MNIST dataset, which can perform digit classification. Instantiate the model by using the NonSpikingNet() constructor. Train the network with the train_many_epochs() method.
Calculate the wall time for training a feed forward neural network model. Record this value in your README.md file. Copy the output of your training run (train and test loss and accuracy) into your README.md as well.
Was there a difference in the accuracy between the two different networks you tested? If so, was it large enough to be notable? With respect to computational efficiency, was there any difference in training time between the two networks? If so, what accounts for this difference?
Activity 3: Examine the Effectiveness of the Spiking Neural Network (50 Points)
The neat thing about spiking neural networks is that we can examine them in a manner that is similar to recording from real neurons. For instance, given a trained model for MNIST, it is possible to look at the activations for a particular neuron over time:
In the above plot, we see that the neuron in the second layer of the network (0 indexed), spiked at around time step 53, and then returned to a resting state.
Using the analysis methods visualize_all_neurons() and visualize_neuron(), look for evidence of success or failure in the classification of the first four MNIST digits in the test set (hint: you can use the following code snippet to grab the data and the labels, assuming SNN is the trained model. data, target = SNN.test_set_loader.__iter__().__next__()) processed via a spiking neural network model. If you have loaded the test set correctly, the digits you will be examining are (in order) 7, 2, 1, 0. With the available analysis methods, you can examine the first or second layer of the network. A plot like the one above may be diagnostic, but other plots that are generated may be useful as well. Given the accuracy you recorded in Activity 1, the network should not be perfect at classifying digits from the MNIST test set, thus it is possible that some neurons are not firing when they should be. Provide your conclusion on what the decision was for the first four test samples in the README.md file, along with the corresponding plots to support your conclusion.
Feedback
If you have any questions, comments, or concerns regarding the course, please
provide your feedback at the end of your README.md.
Submission
To submit your assignment, please commit your work to the homework07 folder
of your homework07 branch in your assignment's GitLab repository:
$ cd path/to/cse-40171-fa19-assignments # Go to assignments repository $ git checkout master # Make sure we are in master branch $ git pull --rebase # Make sure we are up-to-date with GitLab $ git checkout -b homework07 # Create homework07 branch and check it out $ cd homework07 # Go to homework07 directory ... $ $EDITOR README.md # Edit appropriate README.md $ git add README.md # Mark changes for commit $ git commit -m "homework07: complete" # Record changes ... $ git push -u origin homework07 # Push branch to GitLab
Procedure for submitting your work: create a merge request by the process that is described here, but make sure to change the target branch from wscheirer/cse-40171-fa19-assignments to your personal fork's master branch so that your code is not visible to other students. Additionally, assign this merge request to our TA (sabraha2) and add wscheirer as an approver (so all class staff can track your submission).