Code for the Tutorial on Neural Networks

This page contains some code for running the neural networks used in the tutorial. A much better and more detailed introduction to neural networks can be found here (which was the main source for preparing the tutorial and the code below).

Basic Code

This is the basic code for running a neural network. The training data contains 6 vectors. The vectors are positive matches if the first and last elements are 1's. See the other versions for comments explaining the function of each part of the code.

Intermediate Code

This code is slightly more complex so that it is easier to play around with the network. The main difference is that the common settings are moved to parameters at the top of the script. The training data is now randomly generated based on the parameters and the test data is guaranteed to not be in the training set..

Complete Code

This version adds support for further graphical output. It also adds a new training set where the network has to find vectors that contain a certain number of 1's. This is quite complex for this simple network, but it can (sometimes) achieve surprisingly good results with the default parameters. However, slight variation in the default parameters can have large consequences for the performance of the network. Of course, even on the successful runs the network is not learning to count the 1's in the input. However, it is interesting to look at the performance and think about what is missing in order to learn this kind of behaviour.