Question $6[2$ pts$]$: To reduce the risk of neural network overfitting, one solution is to add a

penalty term for the weight magnitude. For example, add a term to the squared error $E$ that

increases with the magnitude of the weight vector. This causes the gradient descent search to

seek weight vectors with small magnitudes, thereby reduces the risk of overfitting. Given a single

layer neural network with $M$ output nodes $($i$.$e$.,$ no hidden layer$),$ assuming the defined squared

error $E$ is defined by

$E(w)=\frac{1}{2}{\displaystyle \underset{n=1}{\overset{N}{}}}{\displaystyle \underset{\text{j}\text{i}\text{n}\text{o}\text{u}\text{t}\text{p}\text{u}\text{t}\text{n}\text{o}\text{d}\text{e}\text{s}}{\overset{?}{}}}[d_j(n)-o_j(n){]}^2+{\displaystyle \underset{i}{\overset{?}{}}},j{w}_{i,j}^2$

Where $N$ denotes the total number of training instances, $d_j(n)$ denotes the desired output of

the $n^{th}$ instance from the $j^{th}$ output node. $o_j(n)$ is the actual output of the $n^{th}$ instance observed

from the $j^{th}$ output node. $w_{i,j}$ is the $i^{th}$ weight value of the $j^{th}$ output node. Assuming an output

node $j$ is using sigmoid activation function $(v_j)=\frac{1}{1+e^{-a{v}_j}}$

Calculate partial derivative of $E(w)$ to weight $w_{i,j}[1pt]$

Derive the weight updating rule for the $i^{th}$ weight of output node $j.[$Hint: use gradient

descent$][1$ pt$]$