Consider an autoencoder with $h$ hidden units, and input

to the autoencoder is a set of $d-$dimensional unlabeled data, $x_i=1^{(i)}{?}^N,$ where $W^1$

denotes the $dh$ weight matrix between the input layer and the hidden layer

and $W^2$ denotes the $hd$ weight matrix between the hidden layer and the output

layer. $f^1$ denotes the activation function for hidden layer, and $f^2$ is the activation

function for the output layer.

$z_j^{(i)}={\displaystyle \underset{k=1}{\overset{d}{}}}{W}_{k,j}^1*x_k^{(i)}$

$a_j^{(i)}=f^1({\displaystyle \underset{k=1}{\overset{d}{}}}{W}_{k,j}^1*x_k^{(i)})$

$t_j^{(i)}={\displaystyle \underset{k=1}{\overset{H}{}}}{W}_{k,j}^2*a_k^{(i)}$

hat$(x{)}_j^{(i)}=f^2({\displaystyle \underset{k=1}{\overset{H}{}}}{W}_{k,j}^2*a_k^{(i)})$

$J(W^1,W^2{)}^{(i)}=||x^{(i)}-$hat$(x{)}^{(i)}|{|}^2={\displaystyle \underset{j=1}{\overset{d}{}}}(x_j^{(i)}-$hat$(x{)}_j^{(i)}{)}^2$

is the reconstruction error for example $x^{(i)}.J(W^1,W^2)={\displaystyle \underset{i=1}{\overset{N}{}}}J(W^1,W^2{)}^{(i)}$ is

the total reconstruction error. We add $1$ to the input and hidden layer so that no

bias term is required.

Compute $\frac{J^{(i)}}{W_{k,i}^2},\frac{J^{(i)}}{z_j^{(i)}},$ and $\frac{J^{(i)}}{W_{k,i}^1}.$