Anaswer question P$1\mathrm{.}3,$ P$2\mathrm{.}3,$ P$3\mathrm{.}1$ and P$3\mathrm{.}2.($Dont write description, just code$).$Also review these codes and find the isuse. Check the following code lines $($DONT USE AI$),$ against these questions and modify if any issue is there.

import numpy as np

import matplotlib.pyplot as plt

from tensorflow import keras

from tensorflow.keras import layers

from tensorflow.keras.datasets import mnist

from tensorflow.keras.models import Sequential

$($x$\_$train, y$\_$train$),($x$\_$test, y$\_$test$)=$ mnist.load$\_$data$()$

x$\_$train $=$ x$\_$train.astype$("$float$32")/255\mathrm{.}0$

x$\_$test $=$ x$\_$test.astype$("$float$32")/255\mathrm{.}0$

x$\_$train $=$ np$.$expand$\_$dims$($x$\_$train, $-1)$

x$\_$test $=$ np$.$expand$\_$dims$($x$\_$test, $-1)$

input$\_$shape $=$ x$\_$train.shape$[1$:$]$

num$\_$classes $=10$

epochs $=1000$

def build$\_$mlp$\_$model$()$:

model $=$ Sequential$([$

layers.Flatten$($input$\_$shape$=$input$\_$shape$),$

layers.Dense$(4,$ activation$=$"relu"$),$

layers.Dense$(4,$ activation$=$"relu"$),$

layers.Dense$(4,$ activation$=$"relu"$),$

layers.Dense$(4,$ activation$=$"relu"$),$

layers.Dense$(4,$ activation$=$"relu"$),$

layers.Dense$(4,$ activation$=$"relu"$),$

layers.Dense$($np$.$prod$($input$\_$shape$),$ activation$=$"sigmoid"$),$

layers.Reshape$($input$\_$shape$)$

$])$

return model

mlp$\_$model $=$ build$\_$mlp$\_$model$()$

mlp$\_$model.compile$($optimizer$="$sgd$",$ loss$=$"mse"$)$

mlp$\_$model.summary$()$

mlp$\_$history $=$ mlp$\_$model.fit$($x$\_$train$[$:$1],$ x$\_$train$[$:$1],$ epochs$=$epochs, batch$\_$size$=1,$ verbose$=1)$

plt$.$plot$($mlp$\_$history.history$[\text{'}$loss$\text{'}])$

plt$.$title$(\text{'}$MLP Training Loss'$)$

plt$.$xlabel$(\text{'}$Epoch$\text{'})$

plt$.$ylabel$(\text{'}$Loss$\text{'})$

plt$.$show$()$

def build$\_$cnn$\_$model$()$:

model $=$ Sequential$([$

layers.Conv$2$D$(10,$ kernel$\_$size$=(5,5),$ activation$=$'relu', padding$=$'same', input$\_$shape$=$input$\_$shape$),$

layers.Conv$2$D$($input$\_$shape$[-1],$ kernel$\_$size$=(1,1),$ activation$=$'sigmoid', padding$=$'same'$)$

$])$

return model

cnn$\_$model $=$ build$\_$cnn$\_$model$()$

cnn$\_$model.compile$($optimizer$="$sgd$",$ loss$=$"mse"$)$

cnn$\_$model.summary$()$

cnn$\_$history $=$ cnn$\_$model.fit$($x$\_$train$[$:$1],$ x$\_$train$[$:$1],$ epochs$=$epochs, batch$\_$size$=1,$ verbose$=1)$

plt$.$plot$($cnn$\_$history.history$[\text{'}$loss$\text{'}])$

plt$.$title$(\text{'}$CNN Training Loss'$)$

plt$.$xlabel$(\text{'}$Epoch$\text{'})$

plt$.$ylabel$(\text{'}$Loss$\text{'})$

plt$.$show$()$

W$1=$ np$.$eye$($input$\_$shape$[0])$

W$2=$ np$.$eye$(4)$

W$3=$ np$.$eye$(4)$

b$1=$ np$.$zeros$((4,))$

b$2=$ np$.$zeros$((4,))$

b$3=$ np$.$zeros$($input$\_$shape$)$

b$3=$ np$.$ones$($input$\_$shape$)*$ x$\_$train$[0][0][0][0]$

def build$\_$identity$\_$cnn$\_$model$()$:

model $=$ keras.Sequential$([$

layers.Conv$2$D$($input$\_$shape$[-1],$ kernel$\_$size$=(1,1),$ activation$=$'linear', padding$=$'same', input$\_$shape$=$input$\_$shape$)$

$])$

return model

def build$\_$identity$\_$relu$\_$cnn$\_$model$()$:

model $=$ keras.Sequential$([$

layers.Conv$2$D$($input$\_$shape$[-1],$ kernel$\_$size$=(1,1),$ activation$=$'relu', padding$=$'same', input$\_$shape$=$input$\_$shape$)$

$])$

return model

Answer these questions:

P$1\mathrm{.}1-$ Implement a fully connected neural network h:$[0,1]^28$x$28($up to$)[0,1]^28$x$28$ model that regresses an image into itself. The architecture should have $7$ trainable dense layers: the first $6$ layers with $4$ neurons and ReLU activation, and an output layer with the necessary number of units and activation.

P$1\mathrm{.}2-$ Train the model using SGD on the appropriate loss function for $10^3$ epochs on the training data. Plot the training loss over epochs.

P$1\mathrm{.}3-$ Plot the prediction over the training set and test set $($you should spot a pattern in the predictions, but since there is some randomness associated with using the GPU we recommend repeating the training $3-5$ times to be sure you pick up the right pattern$).$ Which function do you conjecture h$($x$)$ has learnt $($write it in formula$)?$

P$2\mathrm{.}1-$ Implement a CNN g:$[0,1]^28$x$28($up to$)[0,1]^28$x$28$

model that regresses an image into itself. The architecture should have $2$ convolutional layers: the first with $10$ filters, kernel size $5$x$5$

and the same output size as input, and the second a convolutional output layer with the necessary number of filters, kernel and activation.

P$2\mathrm{.}2-$ Train the model using SGD on the appropriate loss function for $10^3$ epochs on the training data. Plot the training loss over ephocs.

P$2\mathrm{.}3-[$exaclty the same as P$1\mathrm{.}3$ but for g$($x$)]$

P$3\mathrm{.}1-$ Consider a multilayer ReLU network h: R$^$n $($up to$)$ R$^$n such that h$($x$)=$ W$3$ReLU$($W$2$ReLU$($W$1$x$+$b$1)+$b$2)+$b$3$ with W$1($as an element of$)$R$^$a x n$,$ W$2($as an element of$)$ R$^$ n x a$,$ W$3($as an element of$)$ R$^$n x n$,$ b$1($as element of $)$R$^$a; b$2,$ b$3($as element of $)$ R$^$n$.$ Find a possible solution for W$1,$ W$2,$ W$3,$ b$1,$ b$2,$ b$3$ such that h represents the identity funct

What if you want h to represent a constant function that always outputs x$0?$

P$3\mathrm{.}2-$ Consider a CNN g: R$^$n x n $($up to$)$ R$^$n x n model composed by a first hidden convolutional layer with c filters, d x d $($d$>1$ odd$)$ kernel, identity activation and a suitable convolutional output layer. Find a possible architecture for g $($i$.$e$.$ specify the complete architecture, c$,$ the values in the filters, padding and stride$)$ such that g represents the identity function.

If instead of the identity activation, we use a ReLU activation, how should the architecture change?

$($Note: R for set of real