Keras; Convolutional Neural Network $($CNN$)$; ten$-$class classifier for CIFAR$-10$ dataset:

Use cifar$10$ function in keras.datasets to load CIFAR$-10$ dataset. Split it into the training and testing sets. Define a validation set by randomly selecting $20\%$ of the training images along with their corresponding labels. This will be the $$validation$\_$data$.$

Scale the pixel values of the images in all the sets to a value between $0$ and $1.$ Perform this process by dividing the image values with $255.$ Note: No need to flatten the images.

Convert the label vectors for all the sets to binary class matrices using to$\_$categorical$()$ Keras function.

Using Keras library, build a CNN with the following design: $2$ convolutional blocks, $1$ flattening layer,$1$ FC layer with $512$ nodes, and $1$output layer. Each convolutional block consists of two back$-$to$-$back Conv layers followed by max pooling. The filter size is $3$x$3$x image$\_$depth. The number of filters is $32$ in the first convolutional block and $64$ in the second block. Use the following network architecture as a reference:

For students registered with SAS, please contact your instructor for help with this content.

Compile, train, and then evaluate:

Compile the network. Make sure to select a correct loss function for this classification problem. Use Adam optimizer $($Adam$,$ learning rate of $0\mathrm{.}001).$

Use ModelCheckpoint to save the best model based on the lowest validation loss.

Train the network for $50$ epochs with a batch size of $32.$ Remember to assign the validation set to validation$\_$data in the fit function.

Plot the training and validation loss for all the epochs in one plot.

Use the evaluate$()$ Keras function to find the training and validation loss and the accuracy. Report the results.

Now define another model with the same architecture in $($d$)$ and then:

Compile the network. Make sure to select a correct loss function for this classification problem. Use Adam optimizer $($Adam$,$ learning rate of $0\mathrm{.}001).$

Use ModelCheckpoint to save the best model based on the lowest validation loss.

Create an image data generator in Keras for real$-$time data augmentation. The augmentation operations are rotation $(10$ degrees range$),$ width and height shift $(0\mathrm{.}1$ range$),$ and horizontal flip.

Train the network for $50$ epochs with a batch size of $32.$ Remember to use the image data generator. Assign the validation set to validation$\_$data in the fit function.

Plot the training and validation loss for all the epochs in one plot.

Use the evaluate$()$ Keras function to find the training and validation loss and the accuracy. Report the results.

What do you observe from the validation loss in both step $($e$)$ and $($f$)?$ Is the model overfitting or underfitting the training data? Explain.

Now define another model with the same architecture in $($d$),$ except that this time you need to add batch normalization layers to the CNN network.

Add normalization layer after all the convolutional and fully connected layers $($not the output layer$).$ Add them before the activation layers and be noted that there is no need for the bias in the convolutional or fully connected layers.

Compile the network. Make sure to select a correct loss function for this classification problem. Use Adam optimizer $($Adam$,$ learning rate of $0\mathrm{.}01).$

Use ModelCheckpoint to save the best model based on the lowest validation loss.

Train the network for $50$ epochs with a batch size of $64.$ Remember to assign the validation set to validation$\_$data in the fit function.

Plot the training and validation loss for all the epochs in one plot.

Use the evaluate$()$ Keras function to find the training and validation loss and

What do you observe from the training loss in both steps $($e$)$ and $($h$)?$ Explain.