QUESTION $5$

Remember that the softmax function is defined as:

hat$(p{)}_k=\frac{\text{e}\text{x}\text{p}(s_k)}{{\displaystyle \underset{j=1}{\overset{K}{}}}\text{e}\text{x}\text{p}(s_j)}$

where $K$ is the number of classes, $s_k$ is the class score and hat$(p{)}_k$ is the estimated probability of a data sample

belonging to class $k.$

Consider the output layer of a classification network where $K=3,$ and the activation function of the

output layer is the softmax function, as given below.The softmax function in this case is a vector function, which takes a $3-$D vector $)$ as input and

produces a $3-$D vector $(\{$:$[hat(p{)}_1,$hat$(p{)}_2,$hat$(p{)}_3])$ as output.

$($a$)$ Derive the $33$ Jacobian of the softmax function. Obtain an explicit formula for all the elements of

the Jacobian matrix.

$($b$)$ Explain why it is essential to calculate the Jacobian of the softmax.