A softmax layer in a neural network takes an input vector x and produces an output vector y, where 

a. Obtain the derivatives ∂_yj/∂_xi in terms of x-values for the cases i = j and i ≠ j. 

b. Re-express the derivatives in terms of y-values. 

c. What can you say about the signs of the derivatives? 

d. Using the indicator function 1(i = j), which has value 1 if i = j and 0 otherwise, combine your two expressions from part (b) into a single expression for the derivative ∂_yj/∂_xi.

Transcribed Image Text:

yj Σ ear; •k = la €*k