In this assignment, you are asked to train two different MLPs and applies to the created XOR datasets. Additionally, you need to discuss your result. There are total $4$ tasks Answer the question in the designed place and run all the code. For coding parts, ll in your answer in: #Your code goes to here For text answer, ll in your answer in: Your answer goes to here. Answer it in plain tex

$$ Now, in this task, you needs to build another MLP model that consists of two linear layers and given a data x in R$^2,$ we need the following process:

$1.$ transfer it into hidden feature h in R$^100$

$2.$ transfer to h$$ in R$^10$ via linear layer

$3.$ transfer to h$\text{'}\text{'}$ in R via another linear layer

$4.$ convert to output o in $[0,1]$ via sigmoid.

class MultilayerPerceptron$\_$Q$2($torch$.$nn$.$Module$)$:

def $\_\_$init$\_\_($self$)$:

super$($MultilayerPerceptron$\_$Q$2,$ self$).\_\_$init$\_\_()$

#Your code goes to here

def forward$($self$,$ x$)$:

$$#Your code goes to here$$

return None$$

# gradient descend

$$import torch.optim as optim$$

model $=$ MultilayerPerceptron$\_$Q$2()$

op $=$ optim.SGD$($model$.$parameters$(),$lr$=0\mathrm{.}01)$

loss$\_$lst $=[]$

loss$\_$holdout$\_$lst $=[]$

n$\_$epoch $=5000$

for i in range$($n$\_$epoch$)$:$$

# training$$

loss $=$ negative$\_$likeihood$($model$,$ c$1\_$tensor$\_$train, c$2\_$tensor$\_$train$)$ #compute loss $($training data$)$

op$.$zero$\_$grad$()$ #clean cache$$

loss.backward$()$ #compute gradient$$

op$.$step$()$ #gradient descend$$

# vaildation: $($see the performance in unknown data $($unseen by model$))$

with torch.no$\_$grad$()$:$$

loss$\_$holdout $=$ negative$\_$likeihood$($model$,$ c$1\_$tensor$\_$holdout, c$2\_$tensor$\_$holdout$)$ #compute loss $($vaildation$/$holdout data$)$

loss$\_$lst$.$append$($loss$.$item$()/100)$

loss$\_$holdout$\_$lst$.$append$($loss$\_$holdout.item$()/300)$

print$($loss$)$

print$($loss$\_$holdout$)$

Task $4$: Write down your conclusion after visualize the loss change and the decision bound below:$$

plt$.$plot$($np$.$array$($loss$\_$lst$[60$:$]))$

plt$.$plot$($np$.$array$($loss$\_$holdout$\_$lst$[60$:$]))$

# prompt: draw decision boundary of logistic regression$$

# Generate a grid of points for plotting the decision boundary$$

x$\_$min, x$\_$max $=-3,3$

y$\_$min, y$\_$max $=-3,3$

xx$,$ yy $=$ np$.$meshgrid$($np$.$arange$($x$\_$min, x$\_$max, $0\mathrm{.}1),$

np$.$arange$($y$\_$min, y$\_$max, $0\mathrm{.}1))$

# Create a tensor from the grid points$$

grid$\_$tensor $=$ torch.tensor$($np$.$c$\_[$xx$.$ravel$(),$ yy$.$ravel$()],$ dtype$=$torch.float$32)$

# Make predictions for the grid points using the model$$

Z $=$ model$($grid$\_$tensor$)$

Z $=$ Z$.$detach$().$numpy$()$

Z $=$ Z$.$reshape$($xx$.$shape$)$

# Plot the decision boundary$$

plt$.$imshow$($Z$>0\mathrm{.}5)$

plt$.$xlabel$(\text{'}$Feature $1\text{'})$

plt$.$ylabel$(\text{'}$Feature $2\text{'})$

plt$.$title$(\text{'}$Decision Boundary of MLP$\text{'})$

plt$.$legend$()$

plt$.$show$()$

$[$Your answer goes to here. Answer it in plain text$]$