$4.$ This question is on softmax Softmax is briefly described in Jurafsky and Martin, chapter $7,$ section $7\mathrm{.}5\mathrm{.}1.($This whole chapter is a good brief introduction to feedforward neural networks.$)$

$($a$)$ Calculate the probabilities for the following sets of inputs to softmax$($the inputs to softmax are generally called $$logits$)$: $[-2,-1,0,1],[3,3,3,3],[0\mathrm{.}1,0\mathrm{.}2,0\mathrm{.}0,0\mathrm{.}4].$

$($b$)$ If the logits are $[$y$1,$y$2,$y$3],$ are the softmax probabilities the same for $[$y$1+$ c$,$y$2+$ c$,$y$3+$ c$]$ for any value c$?$ Give an argument justifying your answer.

$($c$)$ What would the softmax output be for logits that are $[3\mathrm{.}3,3\mathrm{.}3,-$inf,$-$inf$],$ where $-$inf is the python expression for minus infinity? $($Hint: you don$$t need to do any calculation! Remark: $-$inf is actually used as an input to softmax in the GPT attention mechanism.$)$

$($d$)$ Suppose the predicted probabilities of three possible outcomes $($A$,$ B$,$and C$)$ are $[0\mathrm{.}05,0\mathrm{.}8,0\mathrm{.}1]$ respectively. What is the log$-$loss $($cross entropy loss$)$ if the true outcome $($ie the label in the training set$)$ is A$?$ What is it if the true label is B$?$ and if it is C$?$