Question: Consider again Example 9.4, where we used a softmax output function (S_{L}) in conjunction with the cross-entropy loss: (C(boldsymbol{theta})=-ln g_{y+1}(boldsymbol{x} mid boldsymbol{theta})). Find formulas for

Consider again Example 9.4, where we used a softmax output function \(S_{L}\) in conjunction with the cross-entropy loss: \(C(\boldsymbol{\theta})=-\ln g_{y+1}(\boldsymbol{x} \mid \boldsymbol{\theta})\). Find formulas for \(\frac{\partial C}{\partial \boldsymbol{g}}\) and \(\frac{\partial \boldsymbol{S}_{L}}{\partial \boldsymbol{z}_{L}}\). Hence, verify that:

\[ \begin{equation*} \frac{\partial \boldsymbol{S}_{L}}{\partial \boldsymbol{z}_{L}} \frac{\partial C}{\partial \boldsymbol{g}}=\boldsymbol{g}(\boldsymbol{x} \mid \boldsymbol{\theta})-\boldsymbol{e}_{y+1} \tag{333} \end{equation*} \]
where \(\boldsymbol{e}_{i}\) is the unit length vector with an entry of 1 in the \(i\)-th position.

Step by Step Solution

3.47 Rating (150 Votes )

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock

Direc... View full answer

blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Statistical Techniques in Business Questions!