$2$ Recurrent Neural Networks $[20$ points$]$

Given the LSTM structure in Figure $11$ and the corresponding definition in $(11).$

$([i_t$

$f_t$

$o_t$

$g_t])=([$

$$

$$

$tanh])[([W_1$

$W_2$

$W_3$

$W_4])([h_{t-1}$

$x_t])]$

$c_t=f_{t}o.c_{t-1}+i_{t}o.g_t$

$h_t=o_{t}o.tanh(c_t)$

Let the loss of an LSTM model be $L.$ Assume we have calculated $\frac{\text{d}\text{e}\text{l}\text{L}}{del{i}_{t+1}},\frac{\text{d}\text{e}\text{l}\text{L}}{del{f}_{t+1}},\frac{\text{d}\text{e}\text{l}\text{L}}{del{o}_{t+1}},$

$\frac{\text{d}\text{e}\text{l}\text{L}}{del{g}_{t+1}},\frac{\text{d}\text{e}\text{l}\text{L}}{del{c}_{t+1}}$ and $\frac{\text{d}\text{e}\text{l}\text{L}}{del{h}_{t+1}}.$ Derive gradient formulas for $\frac{\text{d}\text{e}\text{l}\text{L}}{del{i}_t^{}},\frac{\text{d}\text{e}\text{l}\text{L}}{del{o}_t},\frac{\text{d}\text{e}\text{l}\text{L}}{del{g}_t},\frac{\text{d}\text{e}\text{l}\text{L}}{del{c}_t}$ and $\frac{\text{d}\text{e}\text{l}\text{L}}{del{h}_t}$

$($you can assume all gradients are scalars$).$