$h(a)=tanh(a),$ with $tanh(a)=\frac{e^a-e^{-a}}{e^a+e^{-a}}$

A useful feature of this function is that its derivative can be expression in a particularly simple

form:

$h^{\text{'}}(a)=1-h(a{)}^2.$

We consider a standard sum$-$of$-$squares error function, so that the error is given by

$E_n=\frac{1}{2}{\displaystyle \underset{i=1}{\overset{K}{}}}(y_k-t_k{)}^2$

where $y_k$ is the activation of output unit $k,$ and $t_k$ is the corresponding target value, for a

particular input sample $x_n.$

The back propagation algorithm for inferring the parameters $w_{ji}^{(1)}$ and $w_{kj}^{(2)}$ can be described in

the following way. For each sample in the training set in turn, we first perform a forward

propagation using the first two equations above. Next, we compute the error ${}^{\text{'}}$ s for each

output unit using

${}_k=y_k-t_k$

Then, we backpropagate these to obtain the error ${}^{\text{'}}$ s for the hidden units using

${}_j=(1-z_j^2){\displaystyle \underset{i=1}{\overset{K}{}}}{w}_{kj}{}_k$