This experiment is primarily borrowed from Tensorflow's official tutorial available on https://www.tensorflow.org/tutorials/quickstart/beginner .

We will use Tensorflow's Keras API. First open a notebook (.ipynb) and import the following libraries:

 

 

 

from __future__ import absolute_import, division, print_function, unicode_literals

 

 

import tensorflow as tf

import numpy as np

import matplotlib.pyplot as plt

 

 

Now let's load the MNIST dataset. You can either use the code from the previous lab to load the dataset or use the following pre-built method:

 

 

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

x_train, x_test = x_train / 255.0, x_test / 255.0

 

 

Notice that image pixel values are normalized between 0 and 1.

 

Q1) Print out the shapes of the 4 different variables  x_train, y_train, x_test, y_test.

Q2) Split the training data into training (80%) and validation (20%) 

Now let's define the model, we will use tf.Keras module "Sequential". The way Sequential works is that we feed a list of layers and it automatically builds the model sequentially as follows:

 

model = tf.keras.models.Sequential([

  tf.keras.layers.Flatten(input_shape=(28, 28)),

 tf.keras.layers.Dense(128, activation='relu'),

 tf.keras.layers.Dropout(0.2),

 tf.keras.layers.Dense(10)

])

 

As  you can see, each layer is an instance of tf.keras.layers. In the Sequential settings, each layer (except the first layer), takes the output of the previous layer and transforms it based on its specifications.

The first layer  ("tf.keras.layers.Flatten" in our case) takes the input data, that is why we MUST specify the input shape in the first layer.

Let's look at the different layers in the above:

Flatten: reshapes the input from images into a flat vector.

Dense: implements a dense "fully connected: layer.

Dropout: implements a drop-out layer

Q3) Look up the full definitions of these three types of layer in Tensorflow's documentation website. There are some parameters specified in the declaration of each layer, e.g. :128, 0.2, relu ... etc. EXPLAIN what each parameter stand for.

 

Q4) Print the output shape of the model, using

 

 

print(model.output.get_shape().as_list())

 

Now let's train the model, we need to specify the loss criteria, we can use the following loss function

 

 

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

 

Afterwards, we can run the training using the :compile" and "fit" methods of the object "model" 

 

model.compile(optimizer='adam',

             loss=loss_fn,

             metrics=['accuracy'])

model.fit(x_train, y_train, validation_data=(x_val, y_val), epochs=5)

 

 

 

 

Q5) Run the above code, the model will do training, what are the accuracy results you obtain over the validation set?

 

Now let's evaluate the performance over test data. You the following method:

 

 

model.evaluate(x_test, y_test, verbose=2)

 

Q6) What is the model accuracy over the test data.

 

Now we carried out the tutorial on the website. Let's do further experimentation:

 

We will implement another neural with the following architecture:

1. Input layer -> dense layer (output size =200) -> dense layer (output size =200) -> output layer
2. Use ReLU as you activation function

 

Q7) Train the above network for 5 epochs, report the accuracy over validation data and the accuracy over test data.

Answer rating: 100% (QA)

Q1 Printing out the shapes of the variables printShape of xtrain xtrainshape printShape of ytrain yt... View the full answer