Suppose we have a deep MLP classifier and L hidden layers as follows.

hi $=\backslash$sigma $($Wihi$1+$ bi$)$i $=1,...,$ L $(9)$

where the input data vector h$0=$ x in R

D$0\backslash$times $1$ and $\backslash$sigma is the nonlinear activation function. Weights

and the bias vector of the i$-$th layer are Wi in R

Di$\backslash$times Di$1$ and bi in R

Di$\backslash$times $1$

respectively. The readout

function is the softmax. In other words, the probability of the input x belong to the class k is$,$