Homework due Jul $17,202417$:$29$ IST completedDecision Boundaries

$\frac{0}{1}$ point $($graded$)$

In this problem we visualize the "decision boundaries" in $x-$space, corresponding to the four hidden units.

These are the lines in $x-$space where the values of $z_1,z_2,z_3,z_4$ are exactly zero. Plot the decision boundaries

of the four hidden units using the parameters of $W$ provided above.

Enter below the area of the region of your plot that corresponds to a negative $(<mn\; id="EditorElement\_860792">0</mn><mi\; id="EditorElement\_860794">)</mi>$ value for all of the four

hidden units.

You have used $1$ of $3$ attempts

Save

In this problem we will analyze a simple neural network to understand its classification properties. Consider the

neural network given in the figure below, with ReLU activation functions $($denoted by $f)$ on all neurons, and a

softmax activation function in the output layer:

Given an input $x=[x_1,x_2{]}^T,$ the hidden units in the network are activated in stages as described by the

following equations:

$z_1=x_1{W}_{11}+x_2{W}_{11}+W_{01},f(z_1)-$max$\{z_1,0\}$

$z_2-x_1{W}_{18}+x_2{W}_{22}+W_{02},f(z_2)=$max$\{z_2,0\}$

$z_2=x_1{W}_{13}+x_2{W}_{23}+W_{03},f(z_3)=$max$\{z_3,0\}$

$z_4=x_1{W}_{14}+x_3{W}_{24}+W_{04},f(z_4)-$max$\{x_1,0\}$

$u_1=f(x_1)V_{11}+f(z_2)V_{21}+f(x_3)V_{31}+f(z_4)V_{41}+V_{01},f(u_1)=$max$\{u_1,0\}$

$u_2=f(x_1)V_{12}+f(z_2)V_{22}+f(x_3)V_{02}+f(x_4)V_{e2}+V_{02},f(x_2)=$max$\{u_9,0\}.$

The final output of the network is obtained by applying the softmax function to the last hidden layer,

$o_1=\frac{e^{f(u_1)}}{ef(u_1)+e^{f({}_3)}}$

$o_2=\frac{e^{f({}_3)}}{ef({}_1)+ef({}_3)}.$

In this problem, we will consider the following setting of parameters:

$\left[\begin{array}{ccc}{W}_{12}& W_{22}& W_{01}\\ W_{12}& W_{23}& W_{02}\\ W_{13}& W_{23}& W_{03}\\ W_{14}& W_{44}& W_{04}\end{array}\right]=\left[\begin{array}{ccc}1& 0& -1\\ 0& 1& -1\\ -1& 0& -1\\ 0& -1& -1\end{array}\right],$