Assignment $7$ & $8$: Training MLP $$ In this assignment, you are asked to train two different MLPs and applies to the created XOR datasets. Additionally, you need to discuss your result. There are total $4$ tasks Answer the question in the designed place and run all the code. For coding parts, ll in your answer in: #Your code goes to here

For text answer, ll in your answer in: Your answer goes to here. Answer it in plain text Consider a XOR dataset as below import numpy as np #Create XOR dataset. A classical non$-$linear seperatable dataset # Generate data for class A mean$1=[-1,-1]$ cov$1=[[0\mathrm{.}1,0],[0,0\mathrm{.}1]]$ class$0\_$data$1=$ np$.$random.multivariate$\_$normal$($mean$1,$ cov$1,100)$ mean$2=[1,1]$ cov$2=[[0\mathrm{.}1,0],[0,0\mathrm{.}1]]$ class$0\_$data$2=$ np$.$random.multivariate$\_$normal$($mean$2,$ cov$2,100)$ class$0\_$data $=$ np$.$concatenate$(($class$0\_$data$1,$ class$0\_$data$2),$ axis$=0)$ class$0\_$labels $=$ np$.$zeros$(200)$ # Generate data for class B mean$3=[-1,1]$ cov$3=[[0\mathrm{.}1,0],[0,0\mathrm{.}1]]$ class$1\_$data$1=$ np$.$random.multivariate$\_$normal$($mean$3,$ cov$3,100)$ mean$4=[1,-1]$ cov$4=[[0\mathrm{.}1,0],[0,0\mathrm{.}1]]$ class$1\_$data$2=$ np$.$random.multivariate$\_$normal$($mean$4,$ cov$4,100)$ class$1\_$data $=$ np$.$concatenate$(($class$1\_$data$1,$ class$1\_$data$2),$ axis$=0)$ class$1\_$labels $=$ np$.$ones$(200)$ # Combine data and labels X $=$ np$.$concatenate$(($class$0\_$data, class$1\_$data$),$ axis$=0)$ y $=$ np$.$concatenate$(($class$0\_$labels, class$1\_$labels$),$ axis$=0)$ Visualize the data $$ import matplotlib.pyplot as plt plt$.$scatter$($class$0\_$data$[$:$,0],$class$0\_$data$[$:$,1])$ plt$.$scatter$($class$1\_$data$[$:$,0],$class$1\_$data$[$:$,1])$ Start coding or generate with AI$.$ import torch c$1\_$tensor $=$ torch.from$\_$numpy$($class$0\_$data$).$float$()$ c$2\_$tensor $=$ torch.from$\_$numpy$($class$1\_$data$).$float$()$ Task $1$: Training a MLP Model with $10$ dimensional hidden layer $$ Recall the example shown in class can be described as following: Given data $,$ we doing the following process: $1.$ transfer it into hidden feature $2.$ pass through a sigmoid function $3.$ transfer to $4.$ convert to output via sigmoid. Now, in this task, you needs to build a MLP model that consists of two linear layer and given a data $,$ we need the following process: $1.$ transfer it into hidden feature $2.$ pass through a sigmoid function $3.$ transfer to $4.$ convert to output via sigmoid. x in R$2$ h in R$3$ in R h$$ o in $[0,1]$ x in R$2$ h in R$10$ in R h$$ o in $[0,1]$ import torch class MultilayerPerceptron$\_$Q$1($torch$.$nn$.$Module$)$: def $\_\_$init$\_\_($self$)$: super$($MultilayerPerceptron$\_$Q$1,$ self$).\_\_$init$\_\_()$ #Your code goes to here def forward$($self$,$ x$)$: #Your code goes to here return None def negative$\_$likeihood$($model$,$ katydid$\_$tensor, grasshopper$\_$tensor$)$: o $=$ model$($katydid$\_$tensor$)$ #confidence of a data belongs to the katydid likeihood $=-$ torch.log$($o$+0\mathrm{.}000001)$ o $=$ model$($grasshopper$\_$tensor$)$ likeihood$2=-$ torch.log$((1-$o$)+0\mathrm{.}000001)$ overall$\_$loss $=$ likeihood.sum$()+$ likeihood$2.$sum$()$ return overall$\_$loss # randomly hold out samples to evaluate model indices$\_$holdout$\_$c$1=$ torch.randperm$($len$($c$1\_$tensor$))[$:$150]$ indices$\_$holdout$\_$c$2=$ torch.randperm$($len$($c$2\_$tensor$))[$:$150]$ c$1\_$tensor$\_$holdout $=$ c$1\_$tensor$[$indices$\_$holdout$\_$c$1]$ c$2\_$tensor$\_$holdout $=$ c$2\_$tensor$[$indices$\_$holdout$\_$c$2]$ # remove holdout samples from training data $($c$1\_$tensor and c$2\_$tensor$)$ vector$\_$c$1=$ torch.zeros$(200)$ for i in indices$\_$holdout$\_$c$1$: vector$\_$c$1[$i$]=1$ c$1\_$tensor$\_$train $=$ c$1\_$tensor$[$vector$\_$c$1!=1]$ vector$\_$c$2=$ torch.zeros$(200)$ for i in indices$\_$holdout$\_$c$2$: vector$\_$c$2[$i$]=1$ c$2\_$tensor$\_$train $=$ c$2\_$tensor$[$vector$\_$c$2!=1]$ # gradient descend import torch.optim as optim model $=$ MultilayerPerceptron$\_$Q$1()$ op $=$ optim.SGD$($model$.$parameters$(),$lr$=0\mathrm{.}01)$ loss$\_$lst $=[]$ loss$\_$holdout$\_$lst $=[]$ n$\_$epoch $=5000$ for i in range$($n$\_$epoch$)$: # training loss $=$ negative$\_$likeihood$($model$,$ c$1\_$tensor$\_$train, c$2\_$tensor$\_$train$)$ #compute loss $($training data$)$ op$.$zero$\_$grad$()$ #clean cache loss.backward$()$ #compute gradient op$.$step$()$ #gradient descend # vaildation: $($see the performance in unknown data $($unseen by model$))$ with torch.no$\_$grad$()$: loss$\_$holdout $=$ negative$\_$likeihood$($model$,$ c$1\_$tensor$\_$holdout, c$2\_$tensor$\_$holdout$)$ #compute loss $($vaildation$/$holdout data$)$ loss$\_$lst$.$append$($loss$.$item$()/100)$ loss$\_$holdout$\_$lst$.$append$($loss$\_$holdout.item$()/300)$ print$($loss$)$ print$($loss$\_$holdout$)$ Task $2$: Write down your conclusion after visualize the loss change and the decision bound below: $$ plt$.$plot$($np$.$array$($loss$\_$lst$[60$:$]))$ plt$.$plot$($np$.$array$($loss$\_$holdout$\_$lst$[60$:$]))$ # prompt: draw decision boundary of logistic regression # Generate a grid of points for plotting the decision boundary x$\_$min, x$\_$max $=-3,3$ y$\_$min, y$\_$max $=-3,3$ xx$,$ yy $=$ np$.$meshgrid$($np$.$arange$($x$\_$min, x$\_$max, $0\mathrm{.}1),$ np$.$arange$($y$\_$min, y$\_$max, $0\mathrm{.}1))$ # Create a tensor from the grid points grid$\_$tensor $=$ torch.tensor$($np$.$c$\_[$xx$.$ravel$(),$ yy$.$ravel