Hello everyone. This first exercise utilizes the neon framework to classify handwritten digits. While we will go into the specific detail to deep learning algorithms in the next lectures, this exercise will hopefully provide an introduction to the basic concepts that are common to all deep learning models. If you have any questions or issues during the exercise, please post in the forum corresponding to this exercise and someone will help you. All of the exercises run as IPython Jupyter Notebooks, which allow you to interactively create and run Python code. We use Python as it is not only quite popular, but also fast to prototype with, and has a rich ecosystem of existing packages especially, for data science. In this first exercise, we will utilize the computer vision dataset and MNIST, which contains 70,000 images of handwritten digits. Each image is a 28 by 28 pixel handwritten digit from 0-9 as shown here. To classify each image, we will generate a backend, load in this dataset, specify the model architecture, define the training parameters, and then train the model itself. Finally, we will evaluate our first deep learning model. To move between cells and this IPython Notebook, simply press shift and enter. You can always create new cells by selecting insert and insert cell below, and feel free to hack around and modify the code. If you would like to restart the kernel, press kernel and select whichever option you would like. Okay. Let's get started. Because each image is at 28 by 28 pixels, we have a total of 784 features for each image. This will be the input into our neural network. Our model will be a multi-layer perceptron, also called the softmax regression, and will tell us which digit from 0-9 in the image from MNIST corresponds to it. We'll start by setting up the compute backend, which we can do by simply importing gen_ backend from neon and setting the pot size. Remember titch shift and enter to move between the cells. The next step is to load the data, which in many cases can be non-trivial. Many times that will not fit into memory and normally, you would have to specify the training and validation sets. However, in this example, we have included an easy function that downloads the MNIST dataset and loads it into memory. As shown here, the data is partitioned into two datasets: a training dataset and a validation dataset. We will talk in depth about the importances of splitting the dataset, but in essence, we do this so that we can evaluate the model based on data that has not seen before to avoid overfitting. After loading the data, we can start constructing our model. We start by initializing our weights which we will do using a Gaussian distribution with zero mean and 0.01 standard deviation. The initialization function is actually very important to the success of a model and needs to be chosen very carefully. There are many other initializers in neon, many of which we will use in future exercises. Next, we will create the architecture of our model.
