This is a precursor lab to the final project. You will write most of the functions...

 

  
 

Transcribed Image Text:

This is a precursor lab to the final project. You will write most of the functions to use in the final project here. The first thing is to load the MNIST data into our Ma- chine Learning programs. We will set up the digitClassifier.py to expect input as two matrices: (i) the set of training images X and (ii) the set of training labels Y (there will be a corresponding pair of test matrices). Download all 4 files of the MNIST is available at: http://yann.lecun.com/exdb/mnist/ The images in the MNIST data are 28 x 28 pixels, in a proprietory formt For our work we will flatten out each image to a single row of 28 x 28+ 1 = 785 ele- ments in the X matrix. The *+1* is the first element and correspond to the first column of 1's as for any regression X matrix. There are 60,000 example in the train-images-idx3-ubyte.gz file and 60,000 labels in the train-labels-idx1-ubyte.gz file. These are gzipped files. Please read the note (in bold under the data sets) (you may need to replace the '-' after images with a if you get a file-not-found error). The code to load the uncompressed files into your code is given below: # An MNIST loader. import numpy as np #import gzip import struct def load_images (filename): # Open and unzip the file of images: # with gzip.open(filename, 'rb') as f: fh open(filename, 'rb') #uncomment this if you want to use the # Read the header information into a bunch of variables _ignored, n_images, columns, rows = struct. unpack ('>IIII', fh.read(16)) #Read all the pixels into a NumPy array of bytes: all_pixels = np. frombuffer (fh.read(), dtype=np. uint8) # Reshape the pixels into a matrix where each line is an image: return all pixels. reshape (n_images, columns * rows) def stack_ones (X): c1 = np.ones (len (X)) return np.column_stack ((c1, X)) # can use np. shape (X) [0] # The following commands load the data into # 60000 images, each 785 elements (1 bias + 28 * 28 pixels) X_train = stack_ones (load_images ("train-images.idx3-ubyte")) #23Note "-" replace # 60K labels, each with value 1 if the digit is a five, and 0 otherwise Y_train = encode_sevens (load_labels ("train-labels.idx1-ubyte")) # 10000 images, each 785 elements, with the same structure as _train _test = stack_ones (load_images ("t10k-images.idx3-ubyte")) # 10000 labels, with the same encoding as Y_train Y_test = encode_sevens (load_labels ("t10k-labels.idx1-ubyte")) The above file loads all the data into constants X-train etc.. These can be imported into your classification program with import mnist as mn. Write functions: def train( X, Y, numIter, learningRate) This sets up the vector of s and find the s by calling the gradient function. (Recall: the beta vector is updated in each iteration by -gradient(...) *learningRate) def classify (X, beta) which simply rounds and returns the (the pre- dicted result for the logistic regression). def test (X, Y, beta) that computes the percentage of correct classification from: classify (X, beta). (Note the count of correct results are obtained by summing classify (X, beta) ==Y. Recall, Y = 1 for the selected digit and zeros for the rest). The function should print the loss for every tenth iteration. def loss (...) To compute the log-loss function (see Lecture 8 notes). def gradient (X, Y, beta) def sigPredict (X, beta) - (see Lecture 8 Notes) Run train for 100 iterations with a learningRate = 1.e-5 and then 1000 itera- tions with a learning Rate = 1. e-3 to experiment. This is a precursor lab to the final project. You will write most of the functions to use in the final project here. The first thing is to load the MNIST data into our Ma- chine Learning programs. We will set up the digitClassifier.py to expect input as two matrices: (i) the set of training images X and (ii) the set of training labels Y (there will be a corresponding pair of test matrices). Download all 4 files of the MNIST is available at: http://yann.lecun.com/exdb/mnist/ The images in the MNIST data are 28 x 28 pixels, in a proprietory formt For our work we will flatten out each image to a single row of 28 x 28+ 1 = 785 ele- ments in the X matrix. The *+1* is the first element and correspond to the first column of 1's as for any regression X matrix. There are 60,000 example in the train-images-idx3-ubyte.gz file and 60,000 labels in the train-labels-idx1-ubyte.gz file. These are gzipped files. Please read the note (in bold under the data sets) (you may need to replace the '-' after images with a if you get a file-not-found error). The code to load the uncompressed files into your code is given below: # An MNIST loader. import numpy as np #import gzip import struct def load_images (filename): # Open and unzip the file of images: # with gzip.open(filename, 'rb') as f: fh open(filename, 'rb') #uncomment this if you want to use the # Read the header information into a bunch of variables _ignored, n_images, columns, rows = struct. unpack ('>IIII', fh.read(16)) #Read all the pixels into a NumPy array of bytes: all_pixels = np. frombuffer (fh.read(), dtype=np. uint8) # Reshape the pixels into a matrix where each line is an image: return all pixels. reshape (n_images, columns * rows) def stack_ones (X): c1 = np.ones (len (X)) return np.column_stack ((c1, X)) # can use np. shape (X) [0] # The following commands load the data into # 60000 images, each 785 elements (1 bias + 28 * 28 pixels) X_train = stack_ones (load_images ("train-images.idx3-ubyte")) #23Note "-" replace # 60K labels, each with value 1 if the digit is a five, and 0 otherwise Y_train = encode_sevens (load_labels ("train-labels.idx1-ubyte")) # 10000 images, each 785 elements, with the same structure as _train _test = stack_ones (load_images ("t10k-images.idx3-ubyte")) # 10000 labels, with the same encoding as Y_train Y_test = encode_sevens (load_labels ("t10k-labels.idx1-ubyte")) The above file loads all the data into constants X-train etc.. These can be imported into your classification program with import mnist as mn. Write functions: def train( X, Y, numIter, learningRate) This sets up the vector of s and find the s by calling the gradient function. (Recall: the beta vector is updated in each iteration by -gradient(...) *learningRate) def classify (X, beta) which simply rounds and returns the (the pre- dicted result for the logistic regression). def test (X, Y, beta) that computes the percentage of correct classification from: classify (X, beta). (Note the count of correct results are obtained by summing classify (X, beta) ==Y. Recall, Y = 1 for the selected digit and zeros for the rest). The function should print the loss for every tenth iteration. def loss (...) To compute the log-loss function (see Lecture 8 notes). def gradient (X, Y, beta) def sigPredict (X, beta) - (see Lecture 8 Notes) Run train for 100 iterations with a learningRate = 1.e-5 and then 1000 itera- tions with a learning Rate = 1. e-3 to experiment.